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The  primary  purpose  of  the  study  was  to  determine  the  effect  dif- 
ferent forms  of  student  ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .   Part-time  faculty 
opinions  towards  student  ratings  of  instruction  were  also  explored. 

A  quasi-experimental  design  was  used  with  two  experimental  groups 
receiving  different  forms  of  mid-term  student  ratings  feedback, 
printed/oral  consultation  and  printed  only,  and  two  control  groups 
receiving  no  mid-term  student  ratings  feedback.   The  population  con- 
sisted of  94  part-time  faculty;  53  teaching  short-term,  non-credit 
courses  and  41  teaching  long-term,  credit  courses.   Instructor  self- 
ratings  were  also  incorporated  into  the  design. 

The  evaluation  tool  used  to  assess  the  two  dependent  variables, 
student  ratings  of  instruction  and  instructor  self -ratings ,  was  the  In- 
structional Development  and  Effectiveness  Assessment  (IDEA)  standard 
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form.   Results  indicated  that,  generally,  feedback  had  a  modest,  but 
non- significant  effect  on  part-time  faculty  student  ratings  (£<.05). 
When  compared  to  the  printed  only  feedback  condition,  the  printed/oral 
consultation  feedback  condition  appeared  not  to  have  effected  subsequent 
student  ratings  of  part-time  faculty.   Therefore,  similarities  between 
the  two  feedback  conditions  appeared  greater  than  the  differences.   The 
most  significant  change  (2<.05)  in  subsequent  student  ratings  was  found 
in  part-time  faculty  teaching  short-term,  non-credit  courses  in  the  two 
feedback  conditions.  Instructor  self -overrating  and  underrating  by  part- 
time  faculty  seemed  to  produce  a  higher  awareness  of  teaching  practices 
resulting  in  a  change  in  final  instructor  self -ratings .   Post  hoc 
analyses  of  instructor  demographic  variables  relating  to  sex,  educa- 
tional credentials,  and  teaching  experience  found  teaching  experience  to 
be  a  critical  variable  (2<-05)  to  student  opinion  of  instruction. 
Analyses  of  class  motivational  level  found  students  enrolled  in  short- 
term,  non-credit  courses  more  highly  motivated  at  the  .05  level  of  sig- 
nificance . 

Implications  are  that  student  ratings  of  instruction  could  be  util- 
ized as  an  effective  and  appropriate  evaluation  tool  for  part-time 
faculty  teaching  non-credit  courses.  Instructor  self -evaluation  may  be 
used  to  heighten  part-time  faculty  awareness  of  teaching  behavior. 
Recommendations  for  staff  development  applications  are  included. 


ix 


CHAPTER  ONE 
INTRODUCTION 

Adult  part-time  student  participation  in  organized  learning  ac- 
tivities has  increased  significantly  within  the  last  two  decades. 
Boaz's  (1978)  study  indicated  over  five  million  part-time  students  en- 
rolled in  institutions  of  higher  education  nationally  and  a  total  of 
over  21  million  persons  participating  in  adult  education  courses  or 
programs.   In  a  1980  study,  which  included  interviews  of  a  national  rep- 
resentative sample  of  2,000  Americans  25  years  of  age  and  older, 
Aslanian  and  Bricknell  (1980)  showed  that  half  of  the  adult  learners  in- 
terviewed had  studied  at  least  two  different  topics  in  the  past  year. 
Cross  (1980)  has  cited  this  rapid  growth  of  adult  learners  as  a  major 
trend  in  American  higher  education  today  and  an  indication  that  the 
United  States  has  become  a  learning  society. 

The  demand  for  adult  education  is  often  being  met  by  new  learning 
structures,  one  of  which  is  non- credit,  continuing  education  courses, 
taught  by  part-time  instructors  who  are  hired  often  for  their 
"expertise"  in  a  particular  field.   According  to  Bender  and  Hammons 
(1972)  part-timers  bring  something  new  to  the  classroom- -a  breath  of  the 
real  world,  in  the  form  of  day-to-day  experiences"  (p. 21).   Because  of 
their  various  professional  backgrounds,  the  instructors  come  to  the 
classrooms  with  different  experiences  and  expectations  from  full-time 
instructors.   Gaff  (1975)  indicated  that  these  new  learning  structures 
often  impose  a  different  demand  on  learning  and  teaching. 

1 


2 

Since  non-credit  courses  imply  a  non-graded  structure,  part-time 
instructor  teaching  behavior  and  accountability  to  course  content  often 
are  solely  measured  by  student  evaluations  of  instruction.   Research  in- 
dicates that  students  are  capable  of  identifying  and  describing  teaching 
behaviors  which  are  conducive  to  their  learning  environment  (Costin, 
Greenough,  &Menges,  1971;   Feldman,  1976b;   Kulik  &  Kulik,  1974). 
Therefore,  student  opinion  of  instruction  has  become  a  widely  accepted 
and  utilized  evaluation  tool.   Although  the  results  of  student  evalua- 
tions are  intended  to  help  improve  teaching,  the  results  often  are  seen 
only  by  the  instructor.   The  underlying  assumptions  are  that  part-time 
instructors  value  student  opinions,  can  analyze  the  results,  and  will 
utilize  these  results  to  alter  and  improve  their  teaching  behavior. 

Although  there  are  a  variety  of  different  learning  needs  and  expec- 
tations of  both  adult  students  and  part-time  instructors,  few  instruc- 
tional improvement  opportunities  exist  for  faculty  within  institutions 
that  offer  adult  learners  continuing  education  courses.  Two  recurring 
problems  are  the  effectiveness  of  the  evaluation  of  part-time  faculty 
and  the  participation  of  part-time  faculty  in  staff  development  oppor- 
tunities that  do  exist. 

The  expansion  in  the  use  of  part-time  faculty  has  precipitated  the 
adoption  of  various  forms  of  teaching  evaluation  methods  developed  for 
and  utilized  previously  with  full-time  faculty.   One  of  these  evaluation 
methods  is  the  widely-utilized  student  ratings  of  instruction.   Aleamoni 
(1978),  Braunstein,  Klein,  and  Pachla  (1973),  Centra  (1973b),  McKeachie 
and  Linn  (1975),  and  Stevens  and  Aleamoni  (1985)  demonstrated  how  the 
impact  of  student  ratings  of  instruction  can  be  increased  with  the  use 
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of  feedback  to  effect  certain  instructional  changes.   Using  a  meta 
analysis  of  student  ratings  feedback  studies,  Cohen  (1980)  indicated 
that  overall,  instructors  who  received  mid- term  student  ratings  feedback 
averaged  . 16  of  a  ratings  point  higher  on  end- of- term  student  ratings 
than  instructors  who  did  not  recieve  feedback.   Centra  (1973b)  combined 
augmented  student  feedback  with  instructor  self -evaluation.   He 
demonstrated  that  teachers  who  were  "unrealistic"  in  observing  their  own 
behavior  via  self -evaluation  of  instruction  as  compared  to  their 
students'  opinion  of  the  same  instruction  tended  to  make  changes  in 
their  instructional  practices.   Centra  also  found  that  more  instructors 
"change  if  given  information  to  help  them  interpret  scores"  (p. 297).   In 
other  words,  student  ratings  led  to  changes  only  when  teachers  saw  the 
results  in  such  a  way  that  increased  their  impact,  such  as  counsel  from 
a  master  teacher  or  extensive  narrative  evaluations  from  students 
(Centra,  1973b,  McKeachie ,  Linn,  &  Mann,  1971,  Stevens  &  Aleamoni , 
1985).   The  theoretical  justification  behind  Centra's  (1973b)  study  was 
developed  by  Gage,  Runkel,  and  Chaterjee  (1963)  and  may  be  found  in 
equilibrium  theory.   Equilibrium  theorists  assumed  that  when  a  condition 
of  "imbalance"  (Heider,  1958),  or  "dissonance"  (Festinger,  1957,  1964), 
or  "asymmetry"  (Newcomb,  1959)  was  created,  psychological  discomfort  was 
experienced.   This,  in  turn,  "motivated"  the  person  to  reduce  the  dis- 
sonance or  imbalance  and  achieve  consonance  (balance) ,  and  to  avoid 
situations  which  would  increase  dissonance. 

Each  of  these  studies  utilized  full-time  faculty  or  teaching  assis- 
tants. The  improvement  of  teaching  practices  of  part-time  faculty  is  a 
significant  problem  facing  American  higher  education  today.   Therefore, 
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there  is  an  apparent  need  to  look  more  closely  at  the  evaluation  methods 
of  part-time  instruction.   If  the  student  ratings  of  instruction  evalua- 
tion tool  is  an  effective  and  productive  one  for  part-time  faculty, 
educators  can  utilize  this  information  to  change  teaching  behaviors. 
Such  an  investigation  would  help  increase  the  usefulness  of  student  in- 
structional ratings  for  improvement  purposes. 

Statement  of  the  Purpose 

The  primary  purpose  of  this  study  was  to  determine  the  effect  dif- 
ferent forms  of  student -ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .   Part-time  faculty 
opinions  towards  student  ratings  of  instruction  were  also  explored. 

Gage  (1972)  indicated  that  people  acquire  attitudes  and  behaviors 
through  a  process  of  learning,  and  that  knowledge  of  results  or 
"feedback"  is  a  fundamental  condition  of  learning.   Therefore,  such 
knowledge  of  results  can  lead  to  changed  behavior.   Centra  (1973a) 
showed  that  student  feedback  did  effect  some  changes  in  student  ratings 
over  time.   Since  these  findings  suggest  that  feedback  facilitates  be- 
havioral change,  and  behavioral  change  requires  time,  student  ratings 
feedback  should  have  a  more  positive  effect  on  the  instructional  be- 
havior of  part-time  faculty  teaching  longer-term  courses  than  those 
teaching  shorter-term  courses.   Therefore,  the  effectiveness  of  student 
ratings  feedback  as  an  evaluation  tool  for  part-time  faculty  is  the 
focus  of  this  investigation. 

Hypotheses 

This  study  tested  the  following  directional  and  null  hypotheses  at 
the  .05  level  of  significance: 
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1.  Part-time  faculty  who  receive  printed  summary  feedback  along 
with  oral  consultation  about  their  mid- term  student  ratings  will  receive 
higher  end-of-term  student  ratings  than  part-time  faculty  who  receive 
only  printed  summary  feedback  of  mid-term  student  ratings,  and  part-time 
faculty  who  receive  only  printed  summary  feedback  of  mid- term 

student  ratings  will  receive  higher  end-of-term  student  ratings  than 
part-time  faculty  who  do  not  receive  mid-term  student  ratings  feedback. 

2.  Part-time  faculty  assigned  to  both  the  mid- term  feedback  and 
no -feedback  groups  whose  self- ratings  of  instruction  are  higher  than 
their  students'  ratings  of  instruction  at  mid- term  will  receive  higher 
end-of-term  student  ratings  than  part-time  faculty  whose  self-ratings  of 
instruction  are  equal  to  or  lower  than  their  students'  ratings  of  in- 
struction at  mid-term. 

3.  Part-time  faculty  assigned  to  both  the  mid- term  feedback  and 
no-feedback  groups  whose  self-ratings  of  instruction  are  higher  than 
their  students'  ratings  of  instruction  at  mid- terra  will  lower  their  end- 
of-term  self-ratings  of  instruction. 

4.  Part-time  faculty  assigned  to  both  the  mid-terra  feedback  and 
no-feedback  groups  whose  self-ratings  of  instruction  are  equal  to  or 
lower  than  their  students'  ratings  of  instruction  at  raid- terra  will  raise 
their  end-of-term  self-ratings  of  instruction. 

5.  Part-time  faculty  teaching  long-term  courses  who  receive  mid- 
terra  student  ratings  feedback  will  receive  higher  end-of-term  student 
ratings  than  part-time  faculty  teaching  short-term  courses  who  receive 
mid- term  student  ratings  feedback. 
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6.   There  is  no  significant  difference  between  the  opinions  of 
part-time  faculty  teaching  long-term  courses  and  part-time  faculty 
teaching  short-term  courses  towards  student  ratings  of  instruction. 

Need  for  the  Study 

Although  student  ratings  of  instruction  appear  to  be  the  dominant 
evaluation  tool  in  non-credit,  continuing  education  courses  for  adult 
students,  their  influence  on  teaching  behavior  is  still  debatable. 
Since  their  use  is  an  adoption  of  full-time  faculty  evaluation  methods, 
the  appropriateness  of  their  use  with  part-time  faculty  is  in  question. 
Past  research  resulted  in  voluminous  studies  focusing  on  the  reliability 
and  validity  of  student  ratings  as  a  measure  of  instructional  mode. 
Recent  researchers  indicate  increased  attention  directed  toward  enhanc- 
ing the  efficiency  and  effectiveness  of  student  ratings  in  changing  in- 
structional behavior  through  the  use  of  written  and  oral  feedback. 

Because  of  the  widespread  acceptance  and  interest  in  the  use  of 
student  evaluation  of  instruction  data,  several  good  ratings  forms  have 
been  developed.   One  of  these  is  the  Instructional  Development  and 
Effectiveness  Assessment  form  (IDEA)  developed  by  Kansas  State  Univer- 
sity in  1975.   This  instrviment  is  based  on  the  assumption  that  there  are 
different  styles  of  effective  teaching  which  are  dependent  upon  the 
goals  of  the  course  and  the  characteristics  and  motivational  level  of 
the  students.   Hoyt  (1973)  helped  develop  a  questionnaire  format  which 
asks  students  in  various  classes  to  rate  their  progress  on  10  different 
learning  objectives,  describe  the  instructor's  behavior,  and  describe 
different  aspects  of  the  course.   A  separate  form  was  developed  to  col- 
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lect  instructor  ratings  of  the  importance  of  each  of  the  10  learning  ob- 
jectives mentioned. 

Aleamoni  (1978),  Braunstein  et  al .  (1973),  Centra  (1973b),  Cohen 
(1980),  McKeachie  (1969),  and  Tuckman  and  Oliver  (1968)  demonstrated  how 
the  impact  of  student  ratings  of  instruction  can  be  increased  through 
the  use  of  feedback  to  positively  effect  certain  instructional  changes. 
All  of  these  studies  concerned  full-time  faculty  or  teaching  assistants 
teaching  credit  courses.   Therefore,  there  is  an  apparent  need  to  carry 
the  research  one  step  further  in  order  to  determine  the  usefulness  and 
appropriateness  of  student  ratings  of  instruction  with  part-time 

faculty. 

Delimitations 
The  population  for  this  study  was  the  instructors  listed  in  the 
Lifelong  Learning  Catalogue  of  Non-Credit  Courses.  University  of  South 
Florida,  spring,  1986,  and  the  instructors  contracted  to  teach  courses 
in  the  Weekend  College  of  the  Educational  Service  Department  at  Saint 
Leo  College,  Saint  Leo,  Florida.   These  listings  may  or  may  not  have  in- 
cluded all  instructors  of  non-credit  courses,  since  an  instructor's  name 
may  have  been  inadvertently  omitted  from  the  list,  or  an  instructor  may 
have  been  contracted  to  teach  a  course  after  the  catalogue  went  to 
print.   The  list  included  nearly  all  instructors  hired  to  teach  the  non- 
credit  courses  in  the  Division  of  Lifelong  Learning,  School  of  Extended 
Studies,  University  of  South  Florida,  Tampa,  Florida,  and  at  Saint  Leo 
College  in  Saint  Leo,  Florida. 


Due  to  time,  energy,  and  resource  availability  constraints,  the 
following  delimitations  (self-imposed  by  the  researcher)  also  existed  in 
the  study: 

1.  The  researcher  did  not  attempt  to  determine  the  amount  of  stu- 
dent ratings  feedback  desirable  for  instructors  participating  in  the 

s  tudy . 

2.  The  areas  of  feedback  which  should  be  included  to  effect  an  im- 
pact on  teaching  behavior  was  not  determined  in  this  study.   The  basis 
for  change  was  limited  to  the  10  learning  objective  items  and  the  over- 
all evaluation  score  on  the  student  evaluation  of  instruction  form  util- 
ized for  both  self- evaluation  and  student  evaluation  and  the  type  of 
feedback  received. 

3.  No  attempt  was  made  to  analyze  any  possible  long-term  effects 
student  feedback  might  have  had  on  teaching  effectiveness  or  subsequent 
student  ratings. 

4.  The  optimal  level  of  feedback  specificity  to  effect  instruc- 
tional behavioral  changes  on  either  a  short-  or  long-term  basis  was  not 
a  focus  of  this  study. 

5.  Beyond  the  student  ratings  data,  the  researcher  did  not  attempt 
to  investigate  the  teaching  ability  of  the  instructors.   The  extent  of 
change  in  student  reported  teaching  behavior  due  to  self-rating  of  in- 
struction and/or  student  ratings  of  instruction  with  printed  feedback, 
and/or  printed  feedback  with  oral  consultation  was  investigated. 

Limitations 
1.   The  assumption  that  any  change  in  instructors'  teaching  be- 
haviors from  pre-evaluation  to  post-evaluation  was  the  result  of  in- 
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structors  participating  in  a  self-rating,  receiving  printed  student 
feedback,  or  receiving  oral  consultative  feedback,  is  debatable.   Any 
changes  in  teaching  behavior  during  the  pre-evaluation/post-evaluation 
interval  of  the  study  may  have  possibly  occurred  as  a  result  of  other 
moderating  variables  other  than  the  pre -evaluation,  instructor  self- 
rating,  and/or  feedback  variables. 

2.  The  quasi -experimental  design  did  not  allow  the  students  to  be 
randomly  assigned  to  treatment  groups.   The  lack  of  random  student 
assignment  could  have  resulted  in  very  different  or  biased  groups  per 
each  instructor. 

3.  The  generalization  of  the  results  of  the  students  was  limited 
to  part-time  faculty  teaching  non-credit  courses  at  the  University  of 
South  Florida  and  to  part-time  faculty  teaching  credit  courses  in  the 
Weekend  College  of  Saint  Leo  College. 

4.  The  researcher  did  not  attempt  to  analyze  or  categorize  teach- 
ing effectiveness  other  than  that  indicated  by  student  ratings  of  in- 
struction.  Other  measures  of  teaching  effectiveness  could  have  included 
classroom  observations,  video-taping,  peer  evaluation,  etc. 

5.  The  assessment  of  the  importance  given  to  student  evaluations 
by  instructors  was  limited  to  that  of  a  simple  survey  questionnaire. 

6.  Given  time  restraints  and  sample  size  limitations  by  the  re- 
searcher, the  treatments  in  the  study  were  limited  to  two  different 
forms  of  feedback,  printed  data  with  oral  consultation  and  printed  data 
only.   Other  treatment  groups  could  have  included  varying  amounts  of 
personal,  consultative  feedback,  i.e.,  individual  consultative  feedback. 
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Operational  Definitions 

The  following  definitions  were  used  in  this  study: 

Comparative  data  feedback.   A  printed  summary  comparing  item  means 
of  the  instructor's  self -evaluation  data  to  students'  item  means  feed- 
back data  as  measured  by  the  Instructional  Development  and  Effectiveness 
Assessment  standard  form  (Kansas  State  University,  1975). 

Continuing  education.  An  organized,  non-credit  learning  opportunity 
for  adult  students. 

Course  coordinator.   An  employee  of  the  Division  of  Lifelong  Learn- 
ing, School  of  Extended  Studies,  University  of  South  Florida,  respon- 
sible for  coordinating  and  scheduling  non-credit,  continuing  education 
courses  for  adults . 

End-of-term  measure.   The  final  administration  to  all  students 
present  during  the  last  class  meeting  for  both  short-term  and  long-term 
courses  of  the  Instructional  Development  and  Effectiveness  Assessment 
standard  form  (Kansas  State  University,  1975). 

Instructor  Questionnaire.   A  short,  five-item  questionnaire  assess- 
ing instructors'  personal  opinions  of  general  student  rating  evaluation 
tools . 

Instructor  self -evaluation  data.   The  self -assessment  responses  to 
items  on  the  Instructional  Development  and  Effectiveness  Assessment 
(IDEA)  (Kansas  State  University,  1975)  rating  form  of  instructors  teach- 
ing both  long-term  and  short-term  courses. 

Lonp-term  course.   A  14-week  long,  college  credit  course  for  adults 
offered  on  the  weekends  at  various  course  locations  in  the  Tampa  Bay 
area  through  Saint  Leo  College. 
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Mid- term  measure.   The  first  administration  to  all  students  present 
during  the  third  or  fourth  class  meeting  (for  short-term  courses)  and 
the  seventh  class  meeting  (for  long-term  courses)  of  the  Instructional 
Development  and  Effectiveness  Assessment  standard  form  (Kansas  State 
University,  1975). 

Non-credit  course.  An  organized  learning  activity  for  adults  of 
shorter  duration  than  credit  courses  which,  upon  completion,  earns  no 
college  credit. 

Part-time  faculty.   An  instructor  contracted  to  teach  one  or  two 
non-credit,  continuing  education  courses  for  adults  in  the  Division  of 
Lifelong  Learning,  School  of  Extended  Studies,  University  of  South 
Florida  or  one  or  two  credit  courses  offered  on  weekends  through  the 
Weekend  College  of  Saint  Leo  College  during  the  spring  semester,  1986. 

Printed  summary  feedback  with  oral  consultation.   The  computer 
print-out  summary  results  of  the  student  ratings  response  data  to  items 
on  the  Instructional  Development  and  Effectiveness  Assessment  standard 
form  (Kansas  State  University,  1975)  and  comparative  data  of  student 
ratings  to  instructor  self- ratings  as  interpreted  in  an  individual  con- 
ference by  the  course  coordinator  with  the  course  instructor  along  with 
suggestions  for  improvement. 

Printed  summary  feedback.   The  computer  print-out  summary  results 
of  the  student  ratings  response  data  to  items  on  the  Instructional 
Development  and  Effectiveness  Assessment  standard  form  (Kansas  State 
University,  1975)  and  comparative  data  of  student  ratings  to  instructor 
self -ratings  with  printed  instructions  for  self -interpretation. 
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Short-term  course.   Six-  to  eight-week  long,  non-credit,  continuing 
education  course  for  adults  offered  in  the  Division  of  Lifelong  Learn- 
ing, School  of  Extended  Studies,  University  of  South  Florida. 

Student  ratings.   The  mean  score  rating  of  the  10  learning  objec- 
tives and  the  standardized  score  of  the  overall  evaluation  in  Part  I 
Evaluation  Progress  Ratings  section  of  the  IDEA  standard  form  (Kansas 
State  University,  1975). 

Teaching  behavior.   Specific  instructional  behavior  as  measured  by 
items  on  the  Instructional  Development  and  Effectiveness  Assessment 
standard  form  (Kansas  State  University,  1975). 

Organization  of  Remainder  of  the  Research  Report 

The  following  chapters  are  utilized  in  the  remainder  of  the  re- 
search report.   Chapter  Two  includes  the  growth  of  adult  and  continuing 
education  and  the  increased  use  of  part-time  faculty  in  general.   Addi- 
tional research  and  literature  that  were  pertinent  to  the  investigation 
also  are  included  in  this  chapter.   Chapter  Three  contains  the  proce- 
dures utilized  to  test  the  feasibility  of  the  evaluation  instruments, 
the  pilot  study,  as  well  as  the  research  design  and  complete  methodology 
used  in  the  study.   Chapter  Four  contains  the  findings  and  analysis  of 
data.   Chapter  Five  includes  the  summary  of  the  findings  and  the  conclu- 
sions drawn  as  a  result  of  the  study,  as  well  as  implications  for  prac- 
tices and  further  research. 


CHAPTER  TWO 
REVIEW  OF  THE  LITERATURE  AND  RELATED  RESEARCH 

To  acquire  an  accurate  understanding  of  the  relationship  between 
teaching  behavior  and  student  feedback  in  higher  education,  an  examina- 
tion of  relevant  research  is  presented  in  the  following  areas:   (a) 
growth  of  adult  and  continuing  education,  (b)  part-time  faculty  and 
faculty  development  practices,  (c)  adult  development,  (d)  source  of  stu- 
dent feedback,  (e)  nature  and  impact  of  student  feedback,  and  (f)  sum- 
mary and  conclusions.   It  is  acknowledged  that  student  feedback  which  is 
used  for  other  purposes  such  as  pedagogy  and  administrative  decision 
making  also  is  directed  at  changing  teaching  behavior.   This  review, 
however,  focuses  on  the  use  of  student  feedback  for  instructors. 
Growth  of  Adult  and  Continuing  Education 
Social  policy  changes  and  attitudes  during  the  late  1960s  and  early 
1970s  resulted  in  the  opening  wider  of  the  doors  of  American  institu- 
tions of  higher  education  to  an  unprecedented  number  of  non- traditional 
students.   These  open-admission  policies,  stated  Cross  (1980),  resulted 
in  a  significant  increase  in  the  number  of  part-time  adult  learners  who 
"constitute  the  most  rapidly  growing  segment  in  American  education" 
(p. 627).   A  30.8%  growth  in  adult  student  participation  in  organized 
learning  activities  occurred  between  1969  and  1975.   This  increase,  ac- 
cording to  Parson  (1979),  was  more  than  double  the  increase  in  the  adult 
population  during  the  same  time  period.   The  United  States  Bureau  of 
Census  data  indicated  that  in  1981,  21  million  adults  participated  in 
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courses  or  programs  of  one  type  or  another,  representing  13%  of  the  to- 
tal adult  population  in  the  United  States.   The  majority  of  adult  stu- 
dents enrolled  in  these  learning  activities  were  enrolled  on  a  part-time 
basis . 

Part-time  Faculty 

The  literature  reviewed  in  this  and  the  next  two  sections  falls 
into  three  categories:  part-time  faculty,  faculty  development  prac- 
tices, and  adult  development. 

As  institutional  credit  and  non-credit  enrollments  expanded,  demand 
for  increased  numbers  of  college  faculty  continued  to  be  met  through  the 
increased  employment  of  part-time,  adjunct  faculty  at  on-campus  as  well 
as  various  off -campus  locations.   The  perfunctory  practice  of  the  use  of 
part-time  faculty  has  changed  the  staffing  patterns  of  institutions  of 
higher  education.   Bender  and  Hammons  (1972)  reported,  in  a  national 
survey  that  part-time  faculty  composed  40%  of  the  total  122,138  faculty 
members  in  lower  division  colleges  in  the  United  States.   Although 
reasons  for  the  phenomenal  growth  in  the  utilization  of  part-time  in- 
structors have  varied  according  to  individual  institutions,  some  common 
factors  cited  for  justification  included  availability,  flexibility,  ex- 
pertise, salary  cost  differentials,  and  community  relations 
(Friedlander,  1980;  Hammons,  1981;  Kuhns ,  1963;  Lombard! ,  1976). 
Cruise,  Furst,  and  Klimes  (1980)  indicated  that  with  typically  no  health 
insurance,  pension,  or  other  benefits,  part-time  teachers  cost  con- 
siderably less,  no  matter  which  unit  of  output  is  measured.   Spofford 
(1979)  cited  U.S.  Department  of  Labor  estimates  of  nearly  80,000  new 
Ph.D.s  joining  the  ranks  of  100,000  available  adjuncts  as  evidence  of 
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growing  part-time  faculty  labor  availability.   Part-time  faculty 
utilization  allowed  administrators  much  greater  flexibility  with  class 
locations  and  time  schedules  (Friedlander ,  1978;  Kuhns ,  1963;  Lombardi , 
1975)  .   Such  utilization  also  provided  an  important  link  with  local 
governmental  agencies,  community  organizations,  and  industry. 

In  a  1980  study  based  on  the  1975,  1977,  and  1978  Center  for  the 
Study  of  Community  Colleges  (CSCC)  surveys,  Friedlander  explored  teach- 
ing behavior  discrepancies  between  full-time  and  part-time  faculty. 
Significant  differences  were  found  between  part-time  and  full-time  in- 
structors on  most  measures  related  to  instructional  practices.   Specifi- 
cally, part-timers  tended  to  have  less  rigorous  instructional  require- 
ments, less  teaching  experience,  fewer  teaching  credentials,  fewer  out- 
of -class  student  contacts,  and  less  emphasis  on  written  assignments  than 
did  full-time  faculty. 

Cruise  et  al .  (1980)  evaluated  the  instructional  effectiveness  of 
part-timers  by  utilizing  student,  administrator,  and  teacher  self- 
evaluation  instruments.   They  concluded  there  were  no  statistical  dif- 
ferences between  the  instructional  behavior  of  full-timers  and  part- 
timers.  Bender  and  Hammons  (1972),  utilizing  student  judgment  as  their 
criterion,  found  that  "both  the  full-time  and  part-time  faculties 
possessed  the  same  strengths  and  weaknesses  in  their  teaching"  (p. 22). 
Guthrie-Morse  (1981),  however,  expressed  the  possibility  of  qualitative 
differences  in  reference  to  inadequate  part-time  supervision,  evalua- 
tion, commitment,  and  experience. 

As  research  of  part-time  faculty  indicated  qualitative  differences, 
so  too  did  research  of  part-time  student  characteristics  and  their 
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learning  needs.   Aslanian  and  Bricknell  (1980)  indicated  that  adults  are 
motivated  to  learn  following  transitions  from  one  status  in  life  to 
another.   Such  transitions  may  be  caused  by  many  factors,  including 
technological  advancement,  job  re -training,  mandated  education  for 
professional  re-licensure  or  re-certification,  personal  growth  and 
development  needs,  as  well  as  an  increased  acceptance  and  focus  of  the 
lifelong  learning  concept.   Queeney  (1982)  characterized  these  adult 
students  as  a  fairly  elite,  well-educated  group  with  different  learning 
expectations  from  traditional  students,  while  Wolfgang  and  Cowling 
(1981)  demonstrated  significant  differences  between  traditional  age  and 
older  students  in  terms  of  motivation.   Kuh  and  Ardiolo  (1979)  studied 
the  differences  between  traditional  age  and  older  students.   They 
reported  that  the  new,  extremely  diverse  student  body  had  a  more  dif- 
ficult time  adjusting  to  new  situations,  had  feelings  of  inadequacy  and 
self-doubt,  and  preferred  traditional  teaching  methodologies  which 
younger  students  tended  to  reject. 

The  combination  of  changing  student  characteristics  with  different 
learning  needs  and  expectations,  new  educational  settings,  and  instruc- 
tional methods  has  required  different  teaching  practices  and  new  student 
relationships  for  part-time  instructors.   New  learning  structures  such 
as  non-credit,  off -campus,  short-term  courses  have  also  created  new  en- 
vironments with  new  learning  and  teaching  demands  for  part-time  faculty. 
Faculties  must,  therefore,  acquire  new  ideas,  teaching  techniques,  and 
skills  to  meet  these  challenges;  instructional  staff  development  im- 
plications are  eminent. 
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Faculty  Development  Practices 

Effective  teaching  has  been  a  complex  set  of  interrelated  at- 
titudes, knowledge,  skills,  values,  and  motivations  according  to  Gaff 
(1975).   Because  of  the  increased  diversity  among  students,  the  improve- 
ment of  teaching  behavior  and  student  learning  has  necessitated  faculty 
awareness  of  the  complex  interactions  among  students,  institutions,  and 
teachers . 

Many  institutions  of  higher  learning  have  developed  programs  to 
cultivate  and  facilitate  instructional  improvement  through  faculty 
development  in  order  to  meet  these  new  needs  and  challenges.   An  early 
study  by  Miller  and  Wilson  (1963)  indicated  that  the  most  commonly 
reported  development  activities  made  available  to  part-time  faculty 
dealt  with  adjusting  to  rules  and  regulations  of  the  college,  in  other 
words,  a  simple  "orientation  session."   Few  opportunities  were  given  to 
improve  communications  or  teaching  techniques,  other  than  financial  as- 
sistance for  attendance  at  professional  meetings  and  conferences  by  some 
departments.   Very  little  has  been  done  to  enhance  faculty  development 
activities  for  part-time  faculty.   According  to  Moe  (1977)  in  a  study  by 
the  Instructional  ACCtion  Center,  "part-time  faculty  development  was  a 
top  concern"  among  the  institutions  surveyed.   However,  the  most  common 
inservice  activity  made  available  to  part-time  faculty  concerned  the 
rules  and  regulations  of  the  institution.   Smith  (1980)  found  that  some 
of  the  weakest  staff  development  programs  were  those  for  part-time 
faculty. 

Gaff  (1975)  in  a  study  of  faculty  development  practices  among  col- 
leges and  universities  revealed  a  common  set  of  assumptions  upon  which 
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the  general  goal  of  instructional  improvement  rests.   He  reported  that 
three  different  approaches  basically  were  employed:   faculty  develop- 
ment, instructional  development,  and  organizational  development. 

In  the  last  decade  more  attention  has  been  focused  on  part-time 
faculty  development.   Although  several  models  have  been  developed, 
little  emphasis  has  been  expended  on  providing  developmental  assistance 
to  part-time  faculty  on  teaching  strategies  (Leslie,  Kellams,  &  Gunne , 
1982).   Cost  to  the  institution,  lack  of  time  and  interest  or  commitment 
on  the  part  of  the  instructor,  and  great  diversity  among  part-time 
faculty  are  reasons  identified  for  minimal  part-time  faculty  development 
efforts  (Leslie  et  al.,  1982;  Weichenthal,  Means,  &  Kozall,  1977). 

The  uniqueness  of  part-time  instructors  should  be  considered  and 
incorporated  into  development  activity  design  as  well  as  the 
individual's  own  development  needs  and  motivations  (Emmet,  1981).   In- 
creased research  has  resulted  in  the  development  of  new  models  based  on 
the  needs  of  part-time  faculty  (Black,  1981;  Mammons,  1981;  Jamerson, 
1979;  Moe,  1977;  Pedras ,  1984;  Pierce  &  Miller,  1980).   Staff  develop- 
ment topics  for  part-time  faculty  have  included  institutional  mission, 
adult  students,  instructional  development  and  delivery,  evaluation  tech- 
niques, learning  theory,  and  collective  bargaining  (Pedras,  1984). 

Adult  Development 

In  spite  of  the  well -documented  trend  in  the  increase  of  the 
utilization  of  part-time  instructors  by  continuing  education  institu- 
tions and  the  increase  in  staff  development  activities,  considerably 
less  research  attention  has  been  given  to  instructor  motivation  and 
adult  development  needs  as  related  to  changes  in  teaching  behaviors. 
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Previously  most  human  development  studies  have  had  a  child-centered 
orientation  with  no  integrated  theory  to  encompass  total  life  span 
(Neugarten,  1968).   In  recent  years,  however,  there  has  been  an  increase 
in  research  focused  on  adulthood.   In  an  attempt  to  chart  the  progress 
of  adult  development,  gerontological  research  has  steadily  expanded 
knowledge  concerning  middle -age  and  older  adults. 

Of  particular  interest  was  the  research  of  Chickering  (1981),  Erik- 
son  (1972),  Gould  (1979),  Levinson  (1978),  and  Sheehy  (1974).   They  in- 
dicated that  adults,  like  children,  develop  through  several  distinguish- 
able stages  or  transitions.   The  pervasive  similarity  of  this  research 
to  child  development  seemed  to  indicate  that,  even  though  experiencing 
sequential  transitions,  the  quest  for  stability  rather  than  change  was 
the  rule  for  the  remainder  of  adulthood.   A  developmental  and  holistic 
perspective  on  adulthood  allowed  an  appreciation  and  awareness  of  the 
shifting  mix  of  stability  and  change  during  the  life  cycle  (Knox,  1977). 
Based  on  Levinson' s  (1978)  life  transitions,  more  recent  researchers 
have  identified  life  issues  related  to  college  faculty  and  adminis- 
trators.  Faculty  careers  were  associated  with  age  stage  development  by 
Duncan  and  McCombs  (1982). 

Since  basic  learning  theory  held  that  the  learning  of  new  behavior 
was  easier  than  eradicating  earlier  learned  behavior  and  replacing  such 
behaviors  with  others,  results  of  adult  development  research  had  sig- 
nificant implications  for  developing  faculty  and  changing  teaching  be- 
havior.  If  stability  was  preferred  to  change  as  people  grew  older,  then 
a  given  amount  of  change  should  require  more  effort  and  possibly  more 
dynamic  and  powerful  environments  as  a  behavior  or  trait  stabilized. 
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According  to  Sanford  (1973),  for  change  to  occur  at  all  there  needed  to 
be  the  presence  of  "an  appropriate  and  effective  challenge,  one  that  is 
sufficient  to  upset  the  equilibrium,  but  not  so  extreme  as  to  induce 
regression;  in  other  words,  not  too  severe  in  an  objective  sense  and  not 
beyond  the  limits  of  the  individual's  adaptive  capabilities"  (p. 16). 
For  faculty  members,  the  increased  call  for  accountability  and  prevalent 
use  of  student  evaluations  of  instruction  techniques  may  provide  such  a 
challenge . 

Compared  to  the  scope  and  number  of  studies  conducted  regarding  the 
characteristics  and  motives  of  students  and  full-time  instructors  of 
credit  activities,  relatively  few  studies  have  been  conducted  concerning 
the  teaching  characteristics,  motives,  and  effectiveness  of  part-time, 
non-credit,  continuing  education  instructors.   Although  there  has  been  a 
paucity  of  available  data,  there  were  even  fewer  substantive  data  con- 
cerning part-time  instructor  teaching  ability  as  measured  by  student 
ratings  of  instruction.   Consequently,  the  suspicion  many  critics  hold 
of  part-time  faculty  members'  teaching  ability  and  effectiveness  has 
remained  high,  and  continued  questioning  of  their  commitment  to  higher 
education  goals  shall  prevail  until  more  studies  are  conducted  relative 
to  the  teaching  performance  of  part-time  faculty. 
Sources  of  Student  Feedback 

As  student  opinions  of  instruction  became  one  of  the  dominant 
evaluation  tools  in  higher  education,  the  credibility  of  students  as  a 
source  of  information  on  teaching  behavior  has  frequently  and  con- 
tinually been  questioned  by  the  opponents  of  this  form  of  evaluation 
(Hildebrand,  Wilson,  SDienst,  1971;  Page,  1974).   The  argument 
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presented  has  been  that  students  are  not  competent  judges  of  instruction 
because  of  lack  of  experience  and  knowledge,  as  well  as  the  influence  of 
extraneous  factors  to  the  quality  of  teaching  such  as  environmental 
stimuli,  grade  point  average,  and  degree  of  interest  in  the  subject 
matter. 

Doyle  (1975)  indicated  that  whatever  instructor  data  are  gathered 
from  whatever  source  needed  to  be  evaluated  in  terms  of  reliability, 
validity,  generalizability ,  and  utility.   Centra  (1973a),  Costin  et  al . 
(1971),  and  Hildebrand  et  al.  (1971)  all  found  student  ratings  to  be 
reliable  in  terms  of  consistency  and  stability  over  time.   Reliability 
coefficients  in  the  .80s  and  .90s  have  been  obtained  with  consistency 
for  a  class  size  of  20  or  more  students  (Gage,  1974). 

Validity  has  been  defined  as  the  extent  to  which  ratings  measure 
what  they  are  intended  to  measure.   Studies  to  establish  the  validity  of 
student  ratings  have  generally  reported  positive  results  (Aleamoni  & 
Yimer,  1973;  Costin  et  al.,  1971;  Gessner,  1973;  Marsh,  1977;  McKeachie, 
Linn,  &  Mann,  1971;  Sullivan  &  Skanes,  1974).   Shingles  (1977)  reported 
that  procedures  used  to  reduce  the  influence  of  extraneous  factors  do 
not  seem  to  change  instructors'  concerns  about  what  student  ratings  ac- 
tually measure.   On  the  other  hand,  Aubrecht  (1979)  suggested  that  if 
instructors  accept  and  value  knowing  how  satisfied  their  students  are 
and  how  they  perceive  their  teaching  behavior,  student  ratings  are  a 
credible  and  reliable  source  of  information  to  be  utilized.   As  student 
ratings  as  a  data  source  for  information  on  teaching  effectiveness  have 
increased,  so  has  the  body  of  research  literature  in  this  area.   Al- 
though there  remain  concern  and  reseirvation  over  the  use  of  student- 
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ratings  data  as  appropriate  and  sufficient  criteria  for  administrative 
decisions  of  teaching  effectiveness  (Menges,  1979),  their  use  for  in- 
structional improvement  purposes  has  become  an  accepted  practice  (Cohen, 
1980). 

Nature  and  Impact  of  Student  Feedback 

It  generally  has  been  accepted  that  students  are  capable  of  iden- 
tifying and  describing  specific  teaching  behaviors  which  are  conducive 
to  their  learning  environment  (Costin,  et  al . ,  1971;  Feldman,  1976a; 
Kulik  &  Kulik,  1974;  Kulik  &  McKeachie,  1975).   Many  techniques  have 
been  used  over  the  years  to  identify  and  describe  effective  teaching. 
Hildebrand  et  al .  (1971)  surveyed  students  and  faculty  at  the  University 
of  California,  Davis.   Students  were  asked  to  identify  the  worst  and 
best  teachers'  characteristics.   The  most  frequently  mentioned  charac- 
teristics were  such  items  as  ability  to  explain  clearly,  enthusiasm,  in- 
terest in  teaching,  willingness  to  help  students,  friendliness  towards 
students,  knowledge  of  subject,  and  organization.   Many  of  these  same 
qualities  have  been  found  in  similar  studies  by  Crawford  and  Bradshaw 
(1968),  Gaff  and  Wilson  (1971),  and  Perry  (1969).   Studies  such  as  these 
have  been  the  basis  of  items  institutions  utilize  as  an  important  re- 
search and  evalution  tool  of  teaching  behavior  with  predictable 
reliability  and  validity  (Feldman,  1977) . 

If  one  can  agree  with  Gage  (1972)  that  people  acquire  their  at- 
titudes and  behaviors  through  a  process  of  learning  and  that  knowledge 
of  results  or  "feedback"  is  a  fundamental  condition  of  learning,  then  it 
follows  that  such  knowledge  of  results  could  lead  to  changed  behavior. 
This  viewpoint  is  based  on  the  assumption  that  teachers  value  student 
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opinion  enough  to  alter  their  instructional  practices  when  necessary  or 
at  least  their  attitude  towards  change  in  instructional  practices.   This 
assumption  bears  examination,  however. 

Glassman  and  Rotem  (1977)  reviewed  a  large  body  of  research  on  stu- 
dent feedback  and  concluded  that  although  student  ratings  data  may  be 
valuable  for  individual  instructors,  overall,  these  data  have  had  mini- 
mal impact,  at  best,  in  changing  teacher  behavior.   Thomas  (1980)  showed 
that  graduate  student  feedback  to  professors  seemingly  did  not  change 
professors'  teaching  behavior  nor  did  it  significantly  change 
professors'  self -perceptions  of  their  teaching  behavior.   On  the  other 
hand,  Aleamoni  (1978)  and  Stevens  and  Aleamoni  (1985)  all  found  that  in- 
structors who  received  augmented  student  ratings  feedback  improved  on 
certain  teaching  dimensions  over  a  period  of  time.   In  a  more  recent 
critical  review  of  the  research  concerning  college  teaching  improvement, 
Levinson-Rose  and  Menges  (1981)  indicated  that  of  93  studies  reviewed, 
78%  supported  intervention  strategies.   After  a  meta  analysis  of  student 
ratings  feedback,  Cohen  (1980)  showed  that,  in  general,  instructors  who 
received  student  ratings  feedback  averaged  .16  of  a  ratings  point  higher 
on  end-of-term  ratings  than  instructors  who  received  no  feedback.   The 
use  of  student  ratings  consultation  feedback  as  a  tool  for  instructional 
improvement  was  investigated  by  McKeachie  et  al .  (1971).   Their  study 
consisted  of  two  experimental  faculty  groups  (consultation  feedback  of 
student  ratings  and  printed  feedback  of  student  ratings)  and  one  control 
faculty  group  (no  feedback).   Mid-semester  and  end-of -semester  measures 
of  student  ratings  of  instruction  were  compared  to  study  the  effect  of 
consultative  student  ratings  feedback  to  printed  student  ratings  feed- 
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back  on  teaching  effectiveness.   Their  results  indicated  consultation 
enhanced  the  positive  effect  of  the  student  ratings  feedback  and 
resulted  in  subsequently  higher  student  ratings  of  the  participating 
faculty.   Erikson  and  Erikson  (1979)  demonstrated  significant  effects  in 
the  use  of  a  consultation  form  of  augmented  student  ratings  feedback  in 
the  faculty  evaluation  processes.   Based  on  the  analysis  of  22  studies, 
Cohen  (1980)  found  that  student  ratings  feedback  had  a  modest  but  posi- 
tive effect  on  improving  college  instruction  as  measured  by  subsequent 
student  ratings.   Specifically,  a  typical  instructor  who  received  aug- 
mented feedback  performed  at  the  end  of  the  semester  at  the  74th  percen- 
tile.  The  instructor  who  only  received  mid- semester  student  ratings 
performed  at  the  58th  percentile;  the  instructor  receiving  no  feedback 
performed  at  the  50th  percentile.   Aleamoni  (1978)  showed  that  instruc- 
tors who  received  consultation  improved  their  student  ratings  on  at 
least  two  of  five  dimensions  over  a  period  of  one  semester  to  one  year. 
His  design,  however,  utilized  non-equivalent  control  groups  which  raised 
questions  of  threats  to  internal  validity  and  possible  regression  to  the 
mean  (Levinson-Rose  &  Menges ,  1981).   Centra  (1973b)  combined  augmented 
student  feedback  with  instructor  self-evaluation.   He  demonstrated  that 
teachers  who  were  "unrealistic"  in  observing  their  own  teaching  behavior 
via  self -evaluation  of  instruction  as  compared  to  their  students' 
opinion  of  the  same  instruction,  tended  to  make  changes  in  their  in- 
structional practices.   An  additional  finding  of  his  was  that  a  wider 
variety  of  instructors  "change  if  given  information  to  help  them  inter- 
pret scores"  (p. 297).   In  other  words,  student  ratings  led  to  changes 
only  when  teachers  saw  the  results  in  such  a  way  that  increased  their 
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impact,  such  as  counsel  from  a  master  teacher  or  extensive  narrative 
evaluations  from  students  (Centra,  1973b;  McKeachie  et  al.,  1971; 
Stevens  &  Aleamoni ,  1985). 

Centra's  (1973b)  study  was  developed  from  research  by  Gage,  Runkel, 
and  Chaterjee  (1963)  and  may  be  based  in  equilibrium  theory.   Equi- 
librium theorists  assumed  that  when  a  condition  of  "imbalance"  (Heider, 
1958),  or  "dissonance"  (Festinger,  1957,  1964),  or  "asymmetry"  (Newcomb, 
1959)  was  created,  psychological  discomfort  was  experienced.   This,  in 
turn,  "motivated"  the  person  to  reduce  the  dissonance  or  imbalance  and 
achieve  consonance  (balance) ,  and  to  avoid  situations  which  would  in- 
crease dissonance.   When  instructors  receive  student  ratings  feedback 
with  no  help  in  interpretation  or  no  explanation,  ratings  have  provided 
little  help  in  effecting  change.   Accordingly,  it  follows  that  student 
ratings  feedback  or  consequences  of  behavior  must  be  augmented  to  indi- 
cate to  instructors  how  close  their  behavior  approaches  a  given  expecta- 
tion and  then  how  much  and  in  what  direction  they  should  change  their 
behavior  if  they  want  to  come  closer  to  that  behavior,  restore  a  condi- 
tion of  "equilibrium,"  and  utilize  the  data  for  improvement  purposes 
(Cohen,  1980;  Rotem,  1978). 

Summary  of  the  Chapter 

Adult  continuing  education  enrollment  continues  to  increase  at  a 
rapid  rate  with  non- traditional  adult  students  who  come  to  the  classroom 
with  different  learning  levels,  needs,  and  expectations.   Much  of  the 
demand  for  this  type  of  learning  activity  is  being  met  through  the  use 
of  part-time  faculty  at  various  teaching  locations.   While  the  use  of 
part-time  faculty  predominates,  faculty  development  opportunities  for 
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them  to  improve  teaching  effectiveness  remains  minimal.  Accountability 
of  the  effectiveness  of  part-time  instruction  appears  to  be  measured  by 
the  widely-accepted  student  ratings  of  instruction,  which  is  an  evalua- 
tion tool  adapted  from  full-time  faculty  evaluation  systems. 

The  major  implication  of  the  present  review  of  the  research  and  re- 
lated literature  is  that  validity  of  student  opinions  of  instruction  as 
an  instructional  evaluation  tool  is  debatable.   However,  despite  the 
controversy  and  strong  opposition,  student  ratings  of  instruction  still 
have  been  universally  utilized  and  endorsed  by  students  and  educators 
alike.   Feedback  of  student  opinion  of  instruction  has  been  shown  to  be 
a  significantly  effective  tool  in  changing  instructional  behavior 
(Aleamoni,  1978;  Centra,  1973b;  Cohen,  1980;  Levinson-Rose  &  Menges, 
1981;  McKeachie  6e  Linn,  1975;  Rotem,  1978).   Therefore,  if  student  feed- 
back can  possibly  be  made  to  be  more  effective  and  provocative,  a  more 
significant  change  in  teaching  behavior  as  measured  by  student  ratings 
and  other  instruments  or  observational  strategies  may  be  observed. 

It  appears  that  much  attention  has  been  given  to  the  reliability 
and  validity  of  student  ratings  as  a  method  for  evaluating  instructional 
behavior.   Based  on  the  literature  review,  it  is  apparent  that  as  part- 
time  faculty  enter  the  classroom  they  encounter  non- traditional  students 
in  new  educational  settings  with  different  learning  expectations.   Given 
the  significant  differences  in  new  learning  structures  and  teaching 
demands  being  made  of  part-time  faculty  as  compared  to  their  full-time 
counterparts,  there  has  been  a  gap  in  the  research  on  the  impact  of  stu- 
dent ratings  on  instruction  of  part-time  faculty.   While  student  ratings 
feedback  research  has  indicated  that  overall,  feedback  has  had  a  sig- 
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nif leant  effect  on  changing  instructional  behavior,  each  of  these 
studies  involved  full-time  faculty  or  teaching  assistants.   Because  the 
teaching  effectiveness  of  part-time  faculty  has  continued  to  be  measured 
predominantly  with  similar  student  ratings  of  instruction  tools,  it 
therefore  seemed  appropriate  that  an  investigation  be  made  of  the  effect 
of  student  ratings  feedback  on  part-time  faculty.   After  careful 
analysis  of  the  results  of  related  research  studies,  the  researcher  ex- 
pected student  ratings  feedback  to  have  a  significant  effect  on  the  sub- 
sequent student  ratings  of  part-time  faculty. 


CHAPTER  THREE 
METHODOLOGY 

The  primary  purpose  of  this  study  was  to  determine  the  effect  dif- 
ferent forms  of  student  ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .  This  chapter  con- 
tains the  description  of  the  general  research  design,  instrumentation, 
population,  pilot  study,  collection  of  data,  and  analysis  of  data  used 
in  this  investigation. 

In  a  study  of  the  effectiveness  of  student  ratings  feedback  on  col- 
lege instruction.  Centra  (1973b)  utilized  a  three-group  design  which  in- 
cluded an  experimental  faculty  group  (written  feedback) ,  and  two  control 
faculty  groups  (no  feedback  and  posttest  only  with  mid- semester  and  end- 
of-semester  measures).  On  the  basis  of  equilibrium  theory,  a  major 
hypothesis  of  his  was  that  student  ratings  would  produce  changes  in 
teachers  who  had  rated  themselves  more  favorably  than  their  students  had 
rated  them.   Research  results  generally  supported  his  hypothesis. 

McKeachie  and  Linn  (1975)  investigated  the  use  of  student  ratings 
consultation  feedback  for  instructional  improvement.   Their  research 
design  also  consisted  of  two  experimental  faculty  groups  (consultation 
feedback  of  student  ratings  and  printed  feedback  of  student  ratings)  and 
one  control  faculty  group  (no  feedback) .   Results  indicated  an  enhanced 
positive  effect  of  student  ratings  feedback. 
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General  Research  Design 

The  research  design  employed  in  this  investigation  was  a  quasi- 
experimental  design.  This  design  allowed  the  experimenter  to 
combine  the  aspects  of  two  feedback  experimental  groups,  as  found  in 
McKeachie  and  Linn's  (1975)  study,  and  two  control  groups  as  found  in 
Centra's  (1973b)  equilibrium  hypothesis  study. 

The  general  research  design  used  in  this  research  had  a  total  of 
four  instructor  groups  (Gl,  G2 ,  G3 ,  and  G4) .   Two  of  the  four  were  ex- 
perimental groups  which  received  different  forms  of  mid- term  student 
ratings  feedback  (Gl  and  G2) .   As  indicated  in  Table  1,  the  other  two 
groups  were  control  groups  which  received  no  mid- term  student  ratings 
feedback  (G3  and  G4) .   All  four  groups  consisted  of  two  levels  for  each 
group:   instructors  of  long-term  courses  and  instructors  of  short-term 
courses.   Long-term  courses  were  14 -week  credit  courses  taught  by  part- 
time  faculty  in  the  Weekend  College  Division  at  Saint  Leo  College. 
Short-term  courses  were  non-credit  courses  of  six  to  eight  weeks  in 
duration  taught  by  part-time  faculty  in  the  Division  of  Lifelong  Learn- 
ing, School  of  Extended  Studies  at  the  University  of  South  Florida.  All 
instructors  participating  in  the  study  were  randomly  assigned  to  one  of 
the  four  groups.   The  independent  variable  was  mid- term  student  ratings 
feedback  with  two  conditions:   a  full  feedback  condition  (Gl)  consisting 
of  printed  feedback  of  the  mid- term  IDEA  standard  form  summary  report  of 
student  ratings  with  oral  consultation  and  a  partial  feedback  condition 
(G2)  consisting  of  printed  feedback  only  of  mid-term  IDEA  standard  form 
summary  report  of  student  ratings. 
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Table  1 

General  Research  Design 


GROUPS 


TREATMENT 


PRETEST   POSTTEST 


Gl  :  Full  feedback  X 

G2  :  Partial  feedback  X 

G3  :  No  feedback  (pretest  and  posttest  only)  X 

G4  :  No  feedback  (posttest  only) 


Another  factor  incorporated  into  the  research  design  was  instructor 
self -evaluation.   In  Gl,  G2 ,  and  G3  instructor  self -evaluation  data  from 
the  IDEA  standard  form  were  collected  at  mid-term  (pretest)  and  end-of- 
term  (posttest).   Instructor  self- evaluation  data  in  control  G4  were 
collected  at  the  end-of-term  only  (posttest).   The  data  collection  of 
student  ratings  and  instructor  self-ratings  of  instruction  at  the  end  of 
the  term  in  control  G4  was  to  determine  whether  mid- term  ratings  had  a 
sensitizing  effect  on  student  raters  or  instructors. 

Instructor  demographic  characteristics  regarding  sex,  teaching  ex- 
perience, and  educational  degree  obtained  were  collected  by  the  inves- 
tigator prior  to  the  beginning  of  the  study.   These  data  were  collected 
to  investigate  whether  there  was  any  relationship  between  instructor 
characteristics  and  student  ratings  of  teaching  behavior.   The  instruc- 
tor demographic  information  is  presented  in  Table  2.   As  indicated, 
there  was  similar  demographic  distribution  among  the  four  treatment 
groups  involved  in  the  study.   Teaching  experience  and  educational 
degree  obtained  for  one  instructor  in  group  4  were  not  available  to  the 
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Table  2 


Frequency  and  Percentage  Distribution  of  Selected  Instructor  Demographic 
Variables 


VARIABLES 


Gl 
(n=24) 
FREQ.  % 


GROUPS 
G2 
(n=23) 
FREQ.  % 


G3 
(n=23) 
FREQ.  i 


G4 
(n=24) 
FREQ.  s 


Sex 

Male 

Female 

Teaching  Exp.'^ 
0-3  yrs. 
4-9  yrs. 
10-14  yrs. 
15+  yrs. 

Educ.  Degree* 
Bachelor' s 
Master' s 
Doctorate 


14   (15%) 
10   (11%) 


2    (2%) 

17   (19%) 

5    (5%) 


17   (18%) 
6    (6%) 


5  (6%)  3  (3%) 

8  (9%)  12  (13%) 

4  (4%)  4  (4%) 

7  (8%)  4  (4%) 


3    (3%) 

13   (14%) 

7    (7%) 


12   (13%) 
11   (11%) 


2  (2%) 

6  (6%) 

6  (6%) 

9  (10%) 


3    (3%) 

13   (14%) 

7    (7%) 


15   (16%) 
9   (10%) 


4  (4%) 

7  (8%) 

3  (3%) 

9  (10%) 


1    (1%) 

14   (16%) 

8    (9%) 


*The  category  data  for  one  instructor  in  G4  were  not  available. 


searcher.   Of  the  total  number  of  94  instructors  participating  in  the 
study,  62%  were  male  and  38%  were  female,  for  an  approximate  2  to  1 
ratio.   Table  2  also  shows  the  distribution  regarding  teaching  ex- 
perience.  Fifteen  percent  claimed  0-3  years  teaching  experience,  36% 
reported  4-9  years  experience,  while  17%  and  32%  of  the  instructors 
claimed  10-14  years  and  over  15  years  experience,  respectively.   The 
percentage  of  instructors  reporting  a  bachelor's  degree  was  9%,  while  an 
overwhelming  majority  of  63%  held  a  master's  degree,  and  28%  held  a  doc- 
torate degree . 
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Instrumentation 

Centra  (1973a)  demonstrated  that  a  descriptive  evaluation  instru- 
ment allows  the  recipient  to  judge  the  responses  to  items  received  by 
the  evaluators.   Descriptive  feedback  has  been  shown  by  researchers  to 
be  less  threatening  to  the  recipient  and  more  informative  than  judgmen- 
tal feedback  (Harari  &  Zedeck,  1973).   Other  researchers  showed  that 
items  of  inclusion  on  the  instrument  need  to  be  aspects  of  behavior  over 
which  the  recipient  has  control,  specific  behaviors  rather  than  global 
descriptions  or  personality  characteristics,  low  inference  rather  than 
high  inference  items,  and  items  relevant  to  the  specific  teaching  situa- 
tion (Braunstein  et  al.  1973;  Cook,  1979;  Classman,  Killiat,  &  Gmelch, 
1974). 

The  student  ratings  of  instruction  instrument  was  central  to  the 
study  in  measuring  the  perceptions  of  students  regarding  the  teaching 
effectiveness  of  part-time  instructors  and  of  the  instructors'  self- 
perception  of  their  own  teaching  performance.   In  a  critical  review  of 
college  teaching  improvement  utilizing  student  feedback  research, 
Levinson-Rose  and  Menges  (1981)  concluded  that  the  clearest  findings 
were  from  those  studies  which  utilized  discrepancies  between 
instructors'  self -ratings  and  ratings  by  students. 

The  instrument  utilized  in  this  study  was  the  standard  form  of  the 
Instructional  Development  and  Effectiveness  Assessment  (IDEA)  (Kansas 
State  University,  1975).   The  IDEA  evaluation  form  was  first  developed 
by  Donald  Hoyt  in  1968  at  Kansas  State  University  (Hoyt,  1973).   It  be- 
came widely  used  in  Kansas  and  available  since  1975  to  college  instruc- 
tors outside  Kansas  State  University  (Hoyt  &  Cashin,  1977). 
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The  standard  form  of  the  IDEA  survey  instrument  contains  39  items 

(see  Appendixes  A  and  B) .   The  basis  upon  which  the  IDEA  standard  form 

was  developed  is  that  effective  teaching  is  reflected  by  students' 

progress  toward  certain  course  goals.   Therefore,  10  of  the  39  items  on 

the  standard  form  are  course  learning  objectives  which  the  instructor  is 

asked  to  rank  order  in  terms  of  priority.   As  shown  in  Table  3,  these  10 

course  learning  objectives  (items  21-30)  are  contained  in  Part  I 

(Progress  Ratings)  of  the  IDEA  summary  report  and  form  the  basis  for 

evaluating  effective  teaching.   This  section  of  the  report  is  divided 

into  three  sub-categories:   subject  matter  mastery,  development  of 

general  skills,  and  personal  development. 

Table  3 

IDEA  Course  Learning  Objective  Items 

ITEM  CATEGORY 

Part  I.   Evaluation  (Progress  Ratings) 
Subject  Matter  Mastery 

21.  Factual  Knowledge 

22.  Principles  and  Theories 

24.  Professional  Skills  and  Viewpoints 

25.  Discipline's  Methods 

Development  of  General  Skills 

23.  Thinking  and  Problem  Solving 

26.  Creative  Capacities 

29.  Effective  Communication 

Personal  Development 

27.  Personal  Responsibiity 

28.  General  Liberal  Education 

30.  Implications  for  Self -Understanding 

Overall  Evaluation  (Progress  on  Relevant  Objectives) 
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Item  reliabilities  and  standard  errors  of  measurement  for  the  IDEA 
form  were  obtained  by  Cashin  and  Perrin  (1978)  for  three  class  sizes: 
small,  medium,  and  large.   Samples  of  200  or  more  classes  were  obtained 
by  taking  all  small  classes  with  10  raters,  all  medium  classes  with  20 
raters,  and  all  large  classes  with  39,  40,  or  41  raters.   Each  class  was 
split  in  half  and  Spearman- Brown  corrections  were  applied  (see  Appendix 
C) .   An  average  item  reliability  of  .69  was  obtained  for  10  raters,  .81 
for  20,  and  .89  for  40. 

In  developing  the  IDEA  standard  rating  form,  the  standard  errors  of 
measurement  for  the  39  items  on  the  IDEA  survey  form  were  also  calcu- 
lated for  three  class  sizes:   small,  medium,  and  large.   The  average 
standard  error  of  measurement  for  small  classes  was  .37,  .25  for  medium 
classes,  and  .18  for  large  classes  (Cashin  &  Perrin,  1978). 

Validity  of  the  IDEA  evaluation  system  has  been  established  with  a 
data  pool  of  over  23,000  classes.   From  this  pool  a  data  base  for  all 
courses  having  utilized  the  IDEA  system  and  courses  similar  in  motiva- 
tion level  and  class  size  has  been  established  for  evaluative  comparison 
purposes.   Correlation  tables  relating  specific  student  progress  to 
variable  and  specific  teacher  activities  (called  "methods")  are  stored 
in  a  computer  and  compare  student  ratings  of  the  IDEA  system  to  three 
kinds  of  data:   (a)  direct  measures  of  student  learning,  (b)  ratings  by 
others  (than  students),  and  (c)  possible  sources  of  bias  (Aubrecht, 
1979). 

This  instrument  was  modified  by  the  researcher  to  complement  the 
specific  data- gathering  time  frames;  i.e.,  the  mid-term  (pretest)  IDEA 
evaluation  form  items  were  stated  in  the  present  tense  (see  Appendix  A) , 
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while  the  final  (posttest)  IDEA  evaluation  form  remained  unchanged 
utilizing  the  past  tense  (see  Appendix  B) .   In  addition,  two  extra 
global  items  were  included  in  the  evaluation  form  to  assess  the 
students'  overall  evaluation  of  the  instructor  and  the  course  (see  Ap- 
pendixes A  and  B) .   The  additional  two  global  items  were  not  part  of  the 
IDEA  data  pool  and,  therefore,  did  not  affect  the  instrument's  overall 
validity  and  reliability  ratings. 

A  five- item  Instructor  Questionnaire  with  a  five-point  Likert-type 
response  scale  was  developed  by  the  researcher  to  assess  instructor 
opinions  towards  student  ratings  of  instruction  (see  Appendix  D) .   The 
five  items  were  used  to  find  out  how  much  an  instructor  felt  the  in- 
stitution valued  student  opinions  (items  1-2)  and  how  much  he/she  valued 
student  opinions  of  instruction  (items  3-5).   The  questionnaire  was  sent 
to  all  instructors  participating  in  this  study  prior  to  the  starting 
date  of  their  class  along  with  a  cover  memo  (see  Appendix  E) . 

Instructor  information  regarding  the  sex,  teaching  experience,  and 
educational  degree  obtained  of  the  study  participants  was  gathered  by 
the  researcher  prior  to  random  assignment  and  treatment  group  level. 

Pilot  Study 

The  investigator  conducted  a  pilot  study  to  determine  the 
suitability  of  the  five -item  Instructor  Questionnaire  and  the 
feasibility  of  using  the  Instructional  Development  and  Effectiveness 
Assessment  (IDEA)  standard  form  (Kansas  State  University,  1975). 

The  pilot  sample  consisted  of  a  total  of  eight  part-time  faculty 
members  from  the  School  of  Extended  Studies  at  the  University  of  South 
Florida  teaching  short-term  courses.   These  faculty  members  were  ran- 
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domly  selected  and  not  included  in  the  full  study.   Each  subject  was 
randomly  assigned  to  one  of  four  treatment  groups,  and  the  IDEA  evalua- 
tion form  was  administered  during  the  fall  semester,  1985.   As  a  result 
of  the  pilot  administration,  the  IDEA  standard  form  was  modified  to 
reflect  the  specific  data  gathering  time  frame.   As  indicated  in  Appen- 
dixes A  and  B,  the  mid- term  evaluation  form  items  were  stated  in  the 
present  tense,  while  the  end-of-term  form  was  stated  in  the  past  tense. 
The  pilot  study  also  allowed  the  researcher  to  measure  the  amount  of 
time  necessary  to  receive  the  computerized  printed  summary  reports  from 
the  Center  for  Faculty  Evaluation  and  Development,  Kansas  State  Univer- 
sity.  As  a  result,  steps  were  taken  in  consultation  with  the  Center  to 
minimize  the  time  necessary  to  receive  the  reports. 

A  training  program  regarding  uniform  directions  for  IDEA  ad- 
ministration was  developed  by  the  researcher.   Additionally,  the  oral 
consultation  format  regarding  score  interpretation  and  comparative  data 
feedback  utilized  in  the  full  study  was  also  developed  (see  Appendix  F) . 
Both  the  evaluation  administration  training  and  oral  consultation  format 
were  piloted  during  this  time  with  the  research  assistants. 

Present  Study 
Subjects 

The  study  sample  consisted  of  a  total  of  94  part-time  faculty  from 
two  different  post -secondary  institutions  in  Florida.   Forty-one  of  the 
total  participants  were  part-time  instructors  of  long-term,  credit 
courses  contracted  to  teach  one  or  two  Weekend  College  courses  in  the 
Educational  Services  Division  of  Saint  Leo  College.   Of  the  43  part-time 
instructors  informed  of  the  research  study  at  Saint  Leo,  two  declined  to 
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participate,  representing  a  95%  participation  rate  for  this  institution. 
The  other  53  of  the  participants  were  University  of  South  Florida,  part- 
time  instructors  listed  in  the  Lifelong  Learning  Catalogue  of  Non-Credit 
Courses .  spring  semester,  1986,  who  were  teaching  a  course  six  to  eight 
weeks  in  duration  (short-term  courses).   All  part-time  faculty  in  the 
Division  of  Lifelong  Learning  (54  total)  elected  to  participate  in  the 
study.   However,  one  instructor  suffered  ill  health  during  the  third 
week  of  classes  and  was  replaced  by  a  substitute  instructor  for  the 
remainder  of  the  course.   His  student  ratings  and  self- ratings  data  were 
warranted  unusable.   Therefore,  a  total  of  164  evaluation  of  instruction 
measurements  and  164  instructor,  self- evaluation  measurements  were 
analyzed  in  the  study.   The  total  number  of  students  enrolled  in  long- 
term  courses  was  649  and  in  short-term  courses  it  was  897  for  a  total  of 
1,546.   The  total  number  of  students  rating  in  this  study  was  1,017  or 
66%.   Each  instructor  was  randomly  assigned  to  one  of  four  treatment 
groups  by  utilizing  a  table  of  random  numbers. 
Data  Collection 

The  research  was  conducted  during  the  spring  semester,  1986. 
During  this  time,  data  consisting  of  instructor  information,  instructor 
opinion  of  student  ratings,  student  ratings,  and  instructor  pretest  and 
posttest  self-evaluation  of  instruction  ratings  were  collected.   All 
data  were  collected  by  either  the  researcher  or  her  two  assistants.   The 
researcher,  who  was  also  employed  as  a  program  coordinator  at  the 
University  of  South  Florida's  Division  of  Lifelong  Learning,  gave  the 
mid- term  student  ratings  feedback  to  all  instructors  assigned  to  this 
treatment  condition.   The  five -item  Instructor  Questionnaire  that  was 
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developed  by  the  investigator  was  sent  to  each  instructor  participating 
in  this  study  as  part  of  their  spring  semester  course  assignment  let- 
ters.  It  was  accompanied  by  a  memorandum  from  the  Director  of  Lifelong 
Learning  (to  instructors  of  short-term  courses)  and  a  letter  from  the 
researcher  (to  instructors  of  long-term  courses)  indicating  instructions 
for  completion  and  deadlines  for  return  (see  Appendix  E) .   In  both  cases 
the  questionnaire's  value  to  the  institution  was  emphasized  as  an  impor- 
tant part  of  a  research  study  on  student  ratings,  which  the  division  or 
college  was  participating  in  during  the  semester.   The  questionnaire  was 
completed  approximately  two  weeks  prior  to  the  beginning  date  of  each 
course . 

During  the  first  week  of  classes,  a  detailed  letter  from  the  Direc- 
tor of  the  Division  of  Lifelong  Learning  (for  short-term  course 
instructors)  and  the  Dean  of  Educational  Services  of  Saint  Leo  College 
(for  instructors  of  long-term  courses)  was  sent  concerning  the 
division's  or  college's  participation  in  an  important  research  study  ex- 
ploring what  students  are  able  to  evaluate  in  the  classroom  and  how  use- 
ful this  information  might  be  to  the  instructor  (see  Appendix  G) .   These 
letters  emphasized  the  study's  importance  to  the  respective  institution 
and  the  desire  for  all  instructors  to  participate.   It  also  explained 
that  a  new  student  evaluation  form  would  be  utilized  and  that  the  course 
coordinator/researcher  would  be  responsible  for  the  administration  of 
the  form.   Most  classes  had  this  form  administered  twice;  once  at  mid- 
term and  again  at  end-of-term. 

During  the  second  and  third  week  of  classes,  each  instructor  teach- 
ing a  short-term  course  assigned  to  feedback  groups  1  and  2  and  no- 
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feedback  group  3  was  contacted  by  the  course  coordinator/researcher  and 
informed  of  the  mid- term  student  ratings  adminstration  scheduled  for  the 
following  class  meeting.   The  researcher  or  one  of  her  assistants  ad- 
ministered and  collected  the  IDEA  standard  form  at  the  beginning  of  the 
third  or  fourth  class  meeting  from  all  students  present.   The  ad- 
ministrator read  a  statement  stressing  the  importance  of  the  evaluation 
and  its  purpose.   All  students  were  encouraged  to  participate  and  com- 
plete all  41  items  on  the  evaluation  to  the  best  of  their  ability  (see 
Appendix  H) .   The  instructor  was  directed  to  go  to  another  classroom 
during  the  administration  of  the  instrument  and  complete  the  same  IDEA 
standard  form  rating  his/her  own  teaching  behavior  as  he/she  believed 
the  students  would  rate  him/her  and  themselves  thus  far  in  the  term  (see 
Appendix  I).   The  instructor  was  asked  to  prioritize  instructional  ob- 
jectives for  his/her  course  utilizing  the  IDEA  instructor  form  prior  to 
the  mid- term  evaluation  (see  Appendix  J). 

The  IDEA  standard  form  was  again  administered  at  the  beginning  of 
the  sixth,  seventh,  or  eighth  class  meeting  to  all  students  present  in 
all  short-term  courses  participating  in  this  study  (both  feedback  and 
no-feedback  groups).   The  instructor  was  directed  again  to  go  to  another 
classroom  during  the  administration  of  the  instrument  and  asked  to  com- 
plete the  same  IDEA  standard  form  rating  his/her  own  teaching  behavior 
as  he/she  believed  the  students  would  rate  him/her  (see  Appendix  I). 

During  the  mid- term  (pretest)  and  final  (posttest)  administration 
of  the  IDEA  form,  the  students  were  asked  to  code  their  response  cards 
with  the  last  four  digits  of  their  social  security  numbers.   It  was  ex- 
plained that  this  was  for  research  coding  purposes  only,  and  that  the 
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instructor  would  not  see  individual  response  cards,  but  only  the  total 
computer  summary  of  the  student  response  cards.   The  researcher  utilized 
this  procedure  to  reduce  the  possibility  that  any  observed  instructional 
change  was  due  to  possible  attrition  of  low  student  raters  occurring  be- 
tween the  mid  and  final  evaluation  intervals  (see  Appendix  H) . 

The  same  procedure  was  followed  for  each  instructor  teaching  long- 
term  courses.   However,  the  researcher  contacted  each  instructor  and  ad- 
ministered the  mid- term  student  ratings  form  at  the  beginning  of  the 
seventh  class  meeting  to  all  students  present,  and  at  the  fourteenth 
class  meeting  to  all  students  present  for  the  end-of-term  ratings. 
During  each  adminstration,  the  instructor  was  directed  to  go  to  another 
classroom  during  the  administration  of  the  instrument  and  complete  the 
same  form  rating  his/her  own  teaching  behavior  as  he/she  believed  the 
students  would  rate  him/her  (see  Appendix  I).   Again,  the  instructor  was 
asked  to  prioritize  instructional  objectives  for  his/her  course  utiliz- 
ing the  IDEA  instructor  form  prior  to  the  mid-term  evaluation  (see  Ap- 
pendix J)  . 

If  less  than  50%  of  the  total  enrolled  students  were  present  at  the 
last  class  meeting  (sixth,  seventh,  or  eighth  class  meeting  for 
short-term  courses;  fourteenth  for  long-term  courses)  an  IDEA  evaluation 
form  and  a  self-addressed  envelope  was  sent  to  the  absent  students  to 
fill  out  and  return.   This  was  accompanied  by  a  memo  from  the 
coordinator/researcher  (to  short-  and  long-term  term  course  students) 
explaining  the  importance  of  the  students'  evaluation  of  the  instructor 
and  course  (see  Appendix  K) . 


41 

Both  instructors  teaching  short-term  and  long-term  courses  assigned 
to  full  feedback  group,  group  1  (printed  feedback  with  oral 
consultation)  received  an  appointment  date  for  a  consultation  with  the 
course  coordinator/researcher  within  one  week  to  10  days  after  the  stu- 
dent ratings  evaluation  administration.   During  the  approximately  30 
minute  oral  consultation  period,  the  computer-printed  summary  of  the 
students'  rating  of  instruction  was  interpreted  and  explained.   The  in- 
dividual instructor's  self -rating  in  comparison  to  student  feedback  also 
was  presented,  explained,  and  interpreted  (see  Appendix  L) .   Included  in 
the  IDEA  summary  report  were  such  statistics  as  the  mean  for  each  item, 
standard  deviation,  percentage  of  students  in  class  responding,  and  stu- 
dent ratings  on  course  goals  as  compared  to  similar  classes  (see  Appen- 
dix L) .   Suggestions  for  possible  improvement  were  made  where  ap- 
plicable, and  a  paper  regarding  effective  teaching  techniques  and  the 
learning  characteristics  of  adult  students  was  given  to  the  instructor 
(see  Appendix  M) . 

Instructors  assigned  to  the  partial  feedback  group,  group  2 
(printed  feedback  only,  no  consultation),  received  (via  mail)  within  one 
week  to  10  days  after  the  mid- term  student  ratings  administration,  a 
computer  summary  of  the  IDEA  evaluation,  a  comparison  to  the  individual 
instructor's  self-rating  on  the  same  items  (see  Appendix  L) ,  and  a  paper 
describing  the  learning  characteristics  of  adult  students  (see  Appendix 
M)  accompanied  by  a  cover  memo  from  the  researcher  (see  Appendix  N) . 
Standard  interpretation  instructions  were  provided  on  the  back  of  the 
IDEA  summary  form  (see  Appendix  L) .   Included  in  the  report  were  the 
mean  for  each  item,  standard  deviation,  percentage  of  students  in  class 
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responding,  and  student  ratings  data  comparison  to  similar  classes  on 
course  goals  (see  Appendix  L) . 

Instructors  assigned  to  no- feedback  groups  3  and  4  did  not  receive 
any  printed  or  oral  mid- term  or  end-of-term  student  feedback  summaries 
prior  to  the  end  of  this  study. 
Data  Analysis 

The  dependent  variables  in  the  study  were  the  posttest  measures  of 
student  ratings  of  instruction  and  the  posttest  measure  of  instructor 
self -ratings.   The  independent  variable  was  the  type  of  student  ratings 
feedback  received. 

Data  on  the  instructor's  sex,  teaching  experience,  and  educational 
degrees  obtained  were  coded  into  the  statistical  analysis  utilizing  a 
one-way  analysis  of  variance  to  determine  if  there  was  any  relationship 
between  these  variables  and  student  ratings  of  part-time  instruction. 

The  IDEA  standard  form's  basis  for  evaluating  effective  teaching  is 
dependent  upon  how  students  rate  their  progress  on  10  course  learning 
objectives,  each  of  which  is  ranked  by  the  course  instructor  as  either 
essential,  important,  or  of  minor  importance  to  his/her  teaching.   These 
ratings  are  reflected  on  the  first  10  course  objective  items  (item  21 
through  30)  contained  in  Part  I  Evaluation  (Progress  Ratings)  of  the 
IDEA  summary  report  (see  Appendix  L) .   An  additional  overall  evaluation 
score  is  indicated  at  the  bottom  of  Part  I.   Although  no  numbers  are 
reported,  a  verbal  rating  is  given  based  upon  the  weighted  average  of  a 
set  of  T-scores.   Objectives  which  the  instructor  considered  essential 
are  given  double  weights;  important  objectives  are  given  single  weights; 
all  other  objectives  are  given  zero  weights  and  are  dropped  from  the 
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calculations.   For  purpose  of  analysis,  the  researcher  assigned  numeri- 
cal weights  to  the  verbal  T- scores  on  a  1-5  basis  with  1  being  the 
lowest  score  and  5  being  the  highest  score. 

The  following  directional  and  null  hypotheses  were  investigated: 

Hypothesis  1.   Part-time  faculty  who  receive  printed  summary  feed- 
back along  with  oral  consultation  about  their  mid- term  student  ratings 
will  receive  higher  end-of-term  student  ratings  than  part-time  faculty 
who  receive  only  printed  summary  feedback  of  mid- term  student  ratings, 
and  part-time  faculty  who  receive  only  printed  summary  feedback  of  mid- 
term student  ratings  will  receive  higher  end-of-term  student  ratings 
than  part-time  faculty  who  do  not  receive  mid- term  student  ratings  feed- 
back. 

End-of-term  student  rating  means  of  the  10  course  learning  objec- 
tive items  of  Part  I  Evaluation  (Progress  Ratings)  and  the  overall 
evaluation  T- score  of  the  IDEA  standard  form  were  examined  utilizing  a 
one-way  analysis  of  variance  to  test  for  Hypothesis  1.   Multiple  pair- 
wise  comparisons  were  performed  using  Fisher's  Least  Squared  Difference 
procedure  to  identify  specific  group  differences  of  each  of  the  10 
course  learning  objective  item  means  and  the  overall  evaluation  T- score, 
if  significant  differences  were  indicated  (.05  level)  by  the  global 
Anova.   Since  Hypothesis  1  was  directional,  a  one- tailed  test  of 
Fisher's  LSD  procedure  was  used.   To  test  for  sensitivity  effects  of 
pretesting  on  both  the  posttest  student  ratings  and  the  instructor  self- 
ratings,  additional  analyses  of  variance  were  conducted. 

Hypothesis  2.   Part-time  faculty  assigned  to  both  the  mid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
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higher  than  their  students'  ratings  of  instruction  at  mid- term  will 
receive  higher  end-of-term  student  ratings  than  part-time  faculty  whose 
self- ratings  of  instruction  are  equal  to  or  lower  than  their  students' 
ratings  of  instruction  at  mid-term. 

A  linear  regression  model  was  used  on  each  of  the  10  course  learn- 
ing objective  item  means  of  the  student  ratings  and  instructor  self- 
ratings  to  calculate  a  regression  weight.   This  statistical  analysis  is 
similar  to  the  regression  formula  found  in  Centra's  (1973b)  study  which 
was  based  on  equilibrium  theory  and  which  showed  the  relationship  be- 
tween instructor  self- ratings  to  be  linear.   Centra  showed  that  instruc- 
tors who  rated  themselves  higher  than  their  students  rated  them 
(overraters)  and  who  showed  a  greater  discrepancy  (received  feedback) 
also  showed  the  greater  likelihood  of  improvement  on  subsequent  student 
ratings.   The  regression  equation  employed  with  the  feedback  and  no- 
feedback  groups  was  R2  -  al+  bl  Rl  and  c(l-Rl)  where  R2  is  the  predicted 
end-of-term  rating;  thus,  1-Rl  is  the  difference  between  the  instructor 
self -rating  and  the  mid- term  rating.   A  statistically  supported 
hypothesis  would  show  a  significant  difference  between  the  regression 
weights  for  1-R  (i.e.  c)  for  the  feedback  and  no-feedback  groups,  with  c 
for  the  feedback  groups  being  positive  and  greater.   Forty  regression 
weights  were  calculated  and  analyzed  for  differences  utilizing  a  t- 
statistic  on  the  paired  regression  weights.   A  total  of  20  t-statistics 
were  then  subsequently  analyzed  for  differences  utilizing  a  one -tailed 
test. 

Hypothesis  3.   Part-time  faculty  assigned  to  both  the  mid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
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higher  than  their  students'  ratings  of  instruction  at  mid- term  will 
lower  their  end-of-term  self-ratings  of  instruction. 

Hypothesis  4.   Part-time  faculty  assigned  to  both  the  mid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  were 
equal  to  or  lower  than  their  students'  ratings  of  instruction  at  mid- 
term will  raise  their  end-of-term  self-ratings  of  instruction. 

Hypotheses  3  and  4  investigated  the  effect  instructor  self- 
overrating  and  underrating  at  mid- term  had  on  final  instructor  self- 
ratings.   A  correlated  t-test  was  utilized  to  test  these  hypotheses  on 
the  10  course  learning  objective  item  means.   Since  Hypotheses  3  and  4 
were  directional,  a  one-tailed  test  of  the  hypotheses  was  used. 

Hypothesis  5.   Part-time  faculty  teaching  long-term  courses  who 
receive  mid- term  student  ratings  feedback  will  receive  higher  end-of- 
term  student  ratings  than  part-time  faculty  teaching  short-term  courses 
who  receive  mid- term  student  ratings  feedback. 

End-of-term  student  ratings  means  of  the  10  course  learning  objec- 
tive items  of  Part  I  Evaluation  (Progress  Ratings)  and  the  overall 
evaluation  T-score  of  the  IDEA  standard  form  were  examined  utilizing  a 
one-way  analysis  of  variance  to  test  for  Hypothesis  5. 

Hypothesis  6.   There  is  no  significant  difference  between  the 
opinions  of  part-time  faculty  teaching  long-term  courses  and  part-time 
faculty  teaching  short-term  courses  towards  student  ratings  of 
instruction. 

A  two-tailed  related  samples  t-statistical  analysis  was  used  to 
test  for  differences  between  instructor  opinions  towards  student  ratings 
of  instruction  in  Hypothesis  6. 
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An  alpha  level  of  .05  was  used  in  each  case  to  test  the  six 
hypotheses . 

Finally,  a  one-way  Anova  was  used  to  test  for  differences  with  the 
three  instructor  demographic  variables  on  the  10  course  learning  objec- 
tive items  and  the  overall  evaluation  T- score  at  the  .05  level.   Tukey's 
method  was  used  to  test  at  the  .05  level  for  within  group  differences 
for  each  item  means  that  indicated  a  significant  F  value. 

Summary  of  the  Chapter 

This  chapter  contains  the  methodological  procedures  used  in  this 
investigation.   The  student  ratings  instrument  was  described  and 
validity  and  reliability  information  provided.   The  five- item  Instructor 
Questionnaire  was  also  described.   Finally,  this  chapter  contains  a 
description  of  subjects,  data  collection,  and  data  analysis  techniques. 


CHAPTER  FOUR 
FINDINGS 

The  primary  purpose  of  this  study  was  to  determine  the  effect  dif- 
ferent forms  of  student  ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .   Part-time  faculty 
opinions  towards  student  ratings  of  instruction  also  were  explored. 

This  chapter  contains  the  results  of  the  study.   First,  the  results 
pertaining  to  each  of  the  six  hypotheses  under  investigation  are 
described  in  terms  of  either  treatment  condition  or  instructor  level 
(short-term  versus  long-term  courses)  with  reference  to  both  dependent 
variables,  posttest  measures  of  student  ratings  of  instruction,  and  the 
posttest  measures  of  instructor  self-ratings.   Then  the  post  hoc 
analyses  of  the  instructor  demographic  variables  of  the  subjects  and 
student  motivation  levels  are  presented  and  discussed. 

The  study's  research  design  combined  aspects  of  two  feedback  ex- 
perimental groups  as  found  in  McKeachie  and  Linn's  (1975)  augmented 
feedback  study  and  two  no- feedback  control  groups  as  found  in  Centra's 
(1973b)  equilibrium  hypothesis  study. 

The  researcher  pilot  tested  the  instructional  assessment  instru- 
ment, IDEA  standard  form  with  its  41  items  (39  standard  items  and  2  ad- 
ditional items),  and  the  five-item  Instructor  Questionnaire  on  eight  in- 
structors teaching  short-term  courses  at  the  University  of  South  Florida 
during  the  fall  semester,  1985. 
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The  research  involved  165  mid- term  and  final  evaluation  of  instruc- 
tion measurements  and  165  mid-term  and  final  self -evaluation  measure- 
ments that  were  collected  for  the  study.   Of  the  total  population,  two 
instructors  from  Saint  Leo  College  elected  not  to  participate,  while  all 
of  the  part-time  instructors  from  the  Division  of  Lifelong  Learning  at 
the  University  of  South  Florida  participated  in  the  study  for  a  total  of 
95  instructors  in  the  sample.   One  University  of  South  Florida  instruc- 
tor teaching  a  short-term  course  became  critically  ill  and  was  replaced 
with  a  substitute  instructor  to  teach  the  remainder  of  the  course.   His 
mid- term  evaluation  measurements  were,  therefore,  warranted  unusable. 
Therefore,  the  study  sample  ultimately  consisted  of  94  part-time  in- 
structors representing  97%  of  the  total  population.   This  resulted  in  a 
total  of  164  evaluation  of  instruction  measurements  and  164  instructor 
self -evaluation  measurements  analyzed  in  the  study. 

Two  different  measurements  were  used  for  the  dependent  variables. 
The  first  dependent  variable  was  the  posttest  measure  of  student  ratings 
of  instruction.   The  second  dependent  variable  was  the  posttest  measure 
of  instructor  self -ratings .   These  two  variables  were  determined  by  the 
ratings  given  to  10  course  learning  objective  items  and  the  overall 
evaluation  T- score  of  the  IDEA  standard  form  by  all  students  present  at 
the  last  class  meeting  in  all  courses  involved  in  this  study.   Instruc- 
tor self- ratings  of  teaching  behavior  on  the  same  10  course  learning  ob- 
jective items  were  also  collected  at  the  last  class  meeting  from  all  in- 
structors involved  in  the  study.   There  was  no  overall  self -ratings 
score  collected  since  this  score  is  a  weighted  T- score  generated  by  the 
computer,  and  the  instructor  self -ratings  were  not  computer  scored. 
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Therefore,  two  different  dependent  variables  were  independently  measured 
and  analyzed:   (a)  student  ratings  of  instruction  and  (b)  instructor 
self-ratings  of  instruction. 

The  original  intent  of  this  study  was  to  test  six  hypotheses  on  the 
total  sample  of  94  subjects.   The  purpose  of  the  study  was  to  determine 
the  effect  different  forms  of  student  ratings  feedback  had  on  subsequent 
part-time  faculty  student  ratings  and  instructor  self -ratings .   Addi- 
tional considerations  of  interest  developed,  specifically,  differences 
between  motivation  levels  of  students  enrolled  in  credit  and  non-credit 
courses  and  instructor  demographic  variables.   Because  of  these  con- 
cerns, post  hoc  analyses  were  performed  to  measure  the  differences  in 
motivation  levels  and  the  instructor  demographic  variables.   Class 
motivation  levels  between  students  enrolled  in  long-term,  credit  and 
short-term,  non-credit  courses  was  determined  by  the  mean  class  score 
indicated  by  item  36  of  the  IDEA  summary  report  (see  Appendix  L) . 

Descriptive  Analysis 

A  subprogram  of  the  Statistical  Analysis  System,  version  5.08  (SAS, 
1984)  computer  program  was  used  to  calculate  the  Anova,  Fisher's  LSD 
procedure,  t-statistic,  Tukey's  comparisons,  and  regression  weights  of 
the  linear  regression  model.   A  special  computer  program  was  written  to 
calculate  the  t-statistic  of  the  regression  weights  in  the  linear 
regression  model  utilized  for  Hypothesis  2. 

Test  of  Hypothesis  1 

Hypothesis  1.   Part-time  faculty  who  receive  printed  summary  feed- 
back along  with  oral  consultation  about  their  mid- term  student  ratings 
will  receive  higher  end-of-term  student  ratings  than  part-time  faculty 
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who  receive  only  printed  summary  feedback  of  mid- term  student  ratings, 
and  part-time  faculty  who  receive  only  printed  summary  feedback  of  mid- 
term student  ratings  will  receive  higher  end- of- term  student  ratings 
than  part-time  faculty  who  do  not  receive  mid- term  student  ratings  feed- 
back. 

The  overall  Anova  performed  on  each  of  the  10  course  learning  ob- 
jective variables  and  the  overall  evaluation  T- score  revealed  that  there 
were  significant  differences  at  the  .05  level  among  the  four  sets  of 
population  means  on  five  course  learning  objective  items.   The  overall 
evaluation  T-score  showed  no  significant  differences.   The  five  course 
learning  objective  items  which  showed  significant  differences  consisted 
of  the  four  items  related  to  the  subject  matter  mastery  category  and  one 
item  in  the  development  of  general  skills  category  (see  Table  4).   Of 
the  items  found  significant  at  the  .05  level  in  the  subject  matter  mas- 
tery learning  objective  category,  item  21  (factual  knowledge)  had  an  F 
value  of  4.11,  item  22  (principles  and  theories)  had  an  F  value  of  4.95, 
item  24  (professional  skills  and  viewpoints)  had  an  F  value  of  3.66,  and 
item  25  (discipline's  methods)  had  an  F  value  of  2.81.   The  other  item 
showing  a  significant  difference  with  an  F  value  of  3.66  was  item  26 
(creative  capacities) ,  which  is  related  to  the  development  of  general 
skills  category  (see  Appendix  0  for  summary  of  all  Anovas) . 

Further  analysis,  which  utilized  Fisher's  LSD  method  for  planned 
pairwise  comparisons,  indicated  additional  significant  differences  be- 
tween treatment  groups  at  the  .05  level  of  significance  utilizing  a  one- 
tailed  test.   Analysis  of  item  21,  factual  knowledge,  revealed  specific 
significant  differences  at  the  .05  level  between  the  means  of  both  feed- 
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back  groups,  full  feedback  group  (group  1)  and  partial  feedback  group 
(group  2),  and  the  end-of-term  (posttest)  means  of  the  no-feedback  post- 
test  only  group  (group  4)  and  between  the  means  of  the  partial  feedback 
group  (group  2)  and  the  no-feedback  pretest  and  posttest  group  (group 
3).   The  LSD  statistic  for  the  full  feedback  group  (group  1)  was  .413, 
and  the  MSE  for  the  paired  comparisons  was  equal  to  .273,  while  the  de- 
grees of  freedom  equaled  90.   The  LSD  for  the  partial  feedback  group 
(group  2)  was  .482  and  for  the  no- feedback  pretest  and  posttest  group 
(group  3)  it  was  .274.   There  was  no  significant  difference  indicated  at 
the  .05  level  between  the  two  no -feedback  groups,  the  no -feedback 
pretest  and  posttest  group  (group  3)  and  no-feedback  posttest  only  group 
(group  4) . 

Further  analysis  of  item  22  (principles  and  theories)  showed  the 
end-of-term  (posttest)  item  means  of  the  full  feedback  group  (group  1), 
the  partial  feedback  group  (group  2),  and  the  no-feedback  pretest  and 
posttest  group  (group  3)  significantly  different  at  the  .05  level  from 
the  end-of-term  (posttest)  means  of  the  no-feedback  posttest  only  group 
(group  4).   The  MSE  for  the  paired  comparisons  was  .226  and  the  degrees 
of  freedom  was  90.   The  LSD  for  the  full  feedback  group  (group  1)  was 
.363,  for  the  partial  feedback  group  (group  2)  it  was  .518,  and  for  the 
no-feedback  pretest  and  posttest  group  (group  3)  it  was  .262.   Another 
paired  item  comparison  significantly  different  at  the  .05  level  occurred 
between  the  no-feedback  pretest  and  posttest  group  (group  3)  and  the  no- 
feedback  posttest  only  group  (group  4).   The  MSE  for  the  paired  com- 
parison was  .226,  degrees  of  freedom  was  90,  and  the  LSD  for  group  3  was 
.262.   Fisher's  LSD  procedure  on  item  24  (professional  skills  and 


53 

viewpoints)  showed  a  significant  difference  at  the  .05  level  on  the  end- 
of-term  (posttest)  means  of  the  full  feedback  group  (group  1),  partial 
feedback  group  (group  2),  and  no-feedback  pretest  and  posttest  only 
group  (group  3)  and  the  no-feedback  posttest  only  group  (group  4).   The 
MSE  for  the  paired  comparisons  was  .391  and  the  degrees  of  freedom 
equaled  90.   The  LSD  value  for  the  full  feedback  group  (group  1)  was 
.546,  the  LSD  statistic  for  the  partial  feedback  group  (group  2)  was 
.487,  and  the  LSD  for  the  no- feedback  pretest  and  posttest  group  (group 
3)  was  .326.   The  last  item  related  to  subject  matter  mastery  which 
showed  significant  differences  among  the  four  sets  of  population  end-of- 
term  student  ratings  means  was  item  25,  discipline's  methods.   The 
Fisher  LSD  analysis  on  this  item's  group  means  indicated  a  significant 
difference  at  the  .05  level  between  the  end-of-term  (posttest)  student 
ratings  means  of  the  full  feedback  group  (group  1)  and  the  partial  feed- 
back group  (group  2)  and  the  no-feedback  posttest  only  group  (group  4). 
Additionally,  the  difference  between  the  partial  feedback  group  (group 
2)  and  the  no- feedback  pretest  and  posttest  group  (group  3)  was  found 
significant  at  the  .05  level.   The  MSE  for  the  paired  comparisons 
equaled  .470  and  the  degrees  of  freedom  was  equal  to  90.   The  LSD 
statistic  for  the  full  feedback  group  (group  1)  was  .367.   The  LSD  value 
for  the  partial  feedback  group  (group  2)  compared  to  the  no-feedback 
pretest  and  posttest  group  (group  3)  was  .404,  while  it  was  .527  for  the 
no-feedback  posttest  only  group  (group  4)  comparison. 

Further  analysis  utilizing  Fisher's  LSD  procedure  on  the  end-of- 
term  (posttest)  student  ratings  means  of  item  26  (creative  capacities) 
indicated  significant  differences  between  the  full  feedback  group  (group 
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1) ,  partial  feedback  group  (group  2),  no-feedback  pretest  and  posttest 
group  (group  3)  and  the  no-feedback  posttest  only  group  (group  4)  at  the 
.05  level  of  significance.   The  MSE  was  .399  and  the  degrees  of  freedom 
was  90.   The  LSD  statistic  for  the  full  feedback  group  (group  1)  was 
.446,  for  the  partial  feedback  group  (group  2)  it  was  .565,  and  for  the 
no-feedback  mid-term  and  posttest  group  (group  3)  it  was  .438. 

Fisher's  LSD  procedure  results  indicated  no  significant  differences 
on  the  11  items  between  the  no -feedback  pretest  and  posttest  group 
(group  3)  and  the  posttest  only  group  (group  4).   Therefore,  the  mid- 
term evaluation  procedure  did  not  seem  to  have  had  any  sensitizing  ef- 
fects on  the  end-of-term  student  ratings. 

Because  of  the  overall  Anova  and  follow-up  findings,  an  additional 
analysis  was  used  by  the  researcher  to  test  for  differences  between  the 
means  of  the  mid-term  (pretest)  student  ratings  and  the  end-of-term 
(posttest)  student  ratings  within  three  of  the  four  treatment  groups 
(groups  1,  2,  and  3).   A  related  samples  t-test  of  paired  comparisons 
was  utilized  to  test  for  differences  of  pretest  and  posttest  means 
within  the  full  feedback  group  (group  1),  partial  feedback  group  (group 
2),  and  no-feedback  pretest  and  posttest  group  (group  3).   A  one-tailed 
test  for  differences  was  used.   There  was  no  analysis  for  the  no- 
feedback  posttest  only  group  (group  4)  since  this  group's  treatment  con- 
sisted of  an  end-of-term  (posttest)  only  assessment.   The  results  of  the 
analysis  are  presented  in  Table  5.  As  indicated  in  Table  5,  the  full 
feedback  group  (group  1)  showed  significant  differences  at  the  .05  level 
between  the  mid- term  (pretest)  student  ratings  means  as  compared  to  the 
end-of-term  (posttest)  student  ratings  means  on  6  of  the  10  course 


^ 

o 

n) 

X^ 

•X3 

0) 

(U 

fii 

iH 

i-H 

3 

li. 

;^ 

o 

U-l 

to 

C 

o 

•H 

4-1 

(fl 

■H 

> 

0) 

p 

T) 

!-i 

CO 

•a 

c 

CO 

j-i 

tXi 

Tl 

d 

CO 

CO 

d 

CO 

OJ 

s 

4-1 

CO 

0) 

w 

4J 

CO 

o 

CU 

-a 

c 

CO 

4-1 

CO 

<u 

4-1 

0) 

en 

h 

G 

PU 

3 
O 

<u 

M 

4= 

u 

4J 

^ 

>4-( 

o 

o 

CO 
J3 

CO 

13 

a 

<U 

o 

(U 

en 

fl4 

•H 

1 

M 

o 

CO 

z 

a 

e 

•a 

o 

c 

u 

CO 

TJ 

(U 

M 

V4 

o 

•H 

CO 

CO 

^ 

CU 

4-( 

OJ 

o 

tl< 

CD 

iH 

4J 

CO 

<-i 

•H 

3 

4-> 

CO 

M 

(U 

CO 

Pi 

P- 

H 

C/l 

O 

H 

fi. 

1/1 

<^ 

H 
H 

H 

Crt 

tn 

O 

w 

/—S 

P-. 

H 

ro 

W 

CM 

Cri 

II 

P^ 

C 

Ui 

CJ 

H 

< 

uo 

w 

w 

Q 

H 

U 

w 

W 

eii 

lij 

Pi 

irf 

w 

< 

H 
H 

M 

CO 

O 

o 

Ui 

y^N 

Pu 

w 

ro 

fe 

CM 

II 

1-1 

C 

< 

Q  ^ 

w  <r 

flH        II     , 

c 

>J  ^  E 
hJ         c 


55 


00  en  r^        m 

■<3-    CT>   vO  "—1 

iTi  m  ^o        r^ 


CO  o  c» 
r-~  00  in 
vo  m  r^ 


CO 

•  ^ 

T. 

o 

.  0)    en 

u 

rH  x:  >H 

<r  o  1^ 
r-~  r~  r~- 

<■  CO  m 


CO  .-H  in 
CO  00  vo 
<r  CO  in 


O    CO  ^ 

r-H     v£)    VO 

in  -*  in 


o  m  <j\ 
CM  cj\  r^ 

<r  CO  -* 


u  a  '^-l  ^  o 

o  -H  o  a)  CO 

CO  Vj  Vj  -h  -h 

fi<  Cm  pi  >  Q 


vO  CO 
r^  CO 
^  a^ 


ctn  r^  <r 

CNl    -3-    CM 

r-^  CO  r>- 


a\        .-H   ^ 


00    CM    -1 

CO  in  in 
vD  ^  in 


\o  ^o  (^ 

CM    CM    vO 
vO   00    O 


CO  <r 

\0    00 


—(    00 

in  ~a- 
in  00 


r^  rH  r~> 

00   VJ3    CO 

<r  CO  in 


o  w    V4   tn 


Pd 

•H 

•  c 

U 

,-J 

O  D 

O 

■ 

•H 

O 

en 

^    <4-l 

C/3 

u 

c 

O.  rH 

<3> 

(U 

e  aj 

-H 

Ph 

o 

M    W 

CO 

• 

<U 

r~. 

00 

O 

> 

CM 

CM 

CO 

o 

56 
learning  objective  items.   These  significant  differences  were  found  on 
item  21  (factual  knowledge)  with  a  t-statistic  of  2.34,  item  23  (problem 
solving)  had  a  t-statistic  of  2.65,  item  26  (creative  capacities)  had  a 
t-statistic  of  1.84,  and  items  29  (effective  communication),  27 
(personal  responsibility),  and  30  (implications  for  self -understanding) 
had  t-statistics  of  2.60,  2.54,  and  2.67,  respectively.   Each  of  the 
other  four  learning  objective  items,  although  not  significant  at  the  .05 
level,  revealed  an  increased  end-of-term  (posttest)  student  ratings 
mean.   The  overall  evaluation  T-score  was  found  non-significant  at  the 
.05  level. 

The  partial  feedback  group  (group  2)  indicated  a  significant  dif- 
ference at  the  .05  level  of  significance  between  the  mid- term  (pretest) 
student  ratings  mean  and  the  end-of-term  (posttest)  student  ratings 
means  on  three  course  learning  objective  items.   Item  25  (discipline's 
methods)  had  a  t-statistic  of  1.74,  item  28  (general  liberal  education) 
had  a  t-statistic  of  1.85,  and  item  30  (implications  for  self- 
understanding)  had  a  t-statistic  of  3.18.   The  other  seven  course  learn- 
ing objective  items,  although  not  significantly  different  at  the  .05 
level,  indicated  higher  end-of-term  (posttest)  student  ratings  means 
than  mid- term  (pretest)  student  ratings  means.   The  overall  evaluation 
T-score  was  found  significant  with  a  t-statistic  of  1.74. 

The  t-test  evaluation  of  the  no- feedback  pretest  and  posttest  group 
(group  3)  showed  no  significant  differences  between  the  student  ratings 
means  from  mid-term  (pretest)  to  end-of-term  (posttest)  on  any  of  the  10 
course  learning  objective  items  or  the  overall  evaluation  T-score. 
Seven  of  the  learning  objective  items  and  the  overall  evaluation  T-score 
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indicated  higher  or  increased  end-of-term  (posttest)  student  ratings 
means,  while  three  of  the  learning  objective  items  indicated  lower  or 
decreased  end-of-term  (posttest)  means. 

The  initial  Anova  performed  by  the  researcher  on  the  10  course 
learning  objective  items  and  the  overall  evaluation  T- score  indicated 
only  5  out  of  11  student  ratings  means  significantly  different  at  the 
.05  level  of  significance.   Closer  investigation  of  the  between  group 
differences  of  these  significant  items  utilized  Fisher's  LSD  method  for 
follow-up  pairwise  comparisons.   Only  15  learning  course  objective  item 
means  out  of  a  possible  30  paired  item  means  were  found  significant  at 
the  .05  level  using  a  one- tailed  test.   Eleven  of  the  15  significant 
item  means  were  those  items  related  to  subject  matter  mastery  and 
belonging  to  the  two  feedback  groups  (groups  1  and  2) .   Even  though  10 
of  the  15  pairwise  comparisons  of  student  ratings  mean  differences  be- 
tween treatment  groups  were  found  significant  at  the  .05  level  when  the 
full  feedback  group  (group  1)  and  the  partial  feedback  group  (group  2) 
were  each  compared  to  the  no  feedback  posttest  only  group  (group  4) , 
this  finding  was  judged  not  to  be  strong  enough  to  support  Hypothesis  1. 
Nor  could  the  fact  that  55  out  of  a  possible  66  paired  mean  comparisons 
were  in  the  predicted  direction  of  the  hypothesis  be  used  as  a  basis  for 
partially  accepting  Hypothesis  1.   In  fact,  when  the  partial  feedback 
group  (group  2)  was  compared  to  the  full  feedback  treatment  group  (group 
1) ,  8  out  of  11  individual  item  mean  comparisons  were  not  in  the  pre- 
dicted direction  of  the  hypothesis,  suggesting  that  mid- term  partial 
feedback  (printed  feedback)  may  be  at  least  as  effective  as  mid- term 
full  feedback  (printed  feedback  with  oral  consultation).   The  t-test  for 
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related  samples  suggested  that  the  greatest  change  in  student  ratings  of 
instruction  from  mid-term  (pretest)  to  final  evaluation  (posttest)  oc- 
curred within  the  treatment  group  which  received  the  full  feedback 
(printed  feedback  with  oral  consultation).   Comparison  of  mid-term 
(pretest)  to  final  evaluation  (posttest)  means  showed  6  out  of  11  in- 
dividual item  means  significantly  different  at  the  .05  level  of  sig- 
nificance.  The  remaining  five  item  means  for  this  group  were  in  the 
predicted  direction  revealing  increased  final  (posttest)  student  rating 
means . 

Based  on  the  above  mentioned  findings ,  the  results  did  not  appear 
to  be  strong  enough  to  even  partially  support  the  hypothesis  at  the  .05 
level  of  significance.  Therefore,  Hypothesis  1  was  rejected.  However, 
the  full  feedback  and  partial  feedback  groups  (groups  1  and  2)  reported 
consistently  higher  subsequent  student  ratings  than  the  two  no- feedback 
groups  (groups  3  and  4). 

Test  for  Hypothesis  2 

Hypothesis  2.   Part-time  faculty  assigned  to  both  the  mid-term 
feedback  and  no -feedback  groups  whose  self- ratings  of  instruction  are 
higher  than  their  students'  ratings  of  instruction  at  mid- term  will 
receive  higher  end-of-term  student  ratings  than  part-time  faculty  whose 
self- ratings  are  equal  to  or  lower  than  their  students'  ratings  of  in- 
struction at  mid-term. 

The  statistical  analysis  utilized  a  linear  regression  model  to  cal- 
culate the  regression  weights  of  the  10  learning  objective  variables  of 
instructor  self -overrating  and  instructor  self -underrating  in  the  two 
feedback  conditions  and  the  no -feedback  condition.   No  overall  evalua- 
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tion  T-score  was  generated  by  instructor  self -ratings ,  and  thus  was  not 
included  in  this  analysis.   Differences  of  the  40  regression  weights  be- 
tween the  feedback  and  no- feedback  groups  were  tested  using  a  correlated 
t-statistic  and  a  one-tailed  test  of  significance.   Table  6  indicates 
the  results  of  the  t- tests  performed  to  compare  the  regression  weights 
for  instructor  self -underraters  and  overraters  in  the  feedback  and  no- 
feedback  conditions.   Results  for  instructors  who  rated  themselves  less 
favorably  at  mid- term  than  their  students  rated  them  (self -underraters) 
indicated  3  items  (items  27,  28,  and  29)  of  the  10  items  analyzed  were 
significant  at  the  .05  level  for  instructors  who  received  feedback. 
Generally,  the  beta  weights  for  instructor  self -underraters  showed 
fairly  random  differences.   Two  of  the  10  learning  objective  items 
(items  27  and  30)  of  the  instructor  self -overraters  in  the  feedback  con- 
dition were  significant  at  the  .05  level.   Four  other  items  (items  24, 
26,  29,  and  28)  showed  beta  weights  of  the  instructor  overraters  in  the 
feedback  condition  positive  and  greater  than  the  beta  weights  of  those 
in  the  no- feedback  condition.   Generally,  overrating  did  not  appear  to 
be  a  valid  predictor  of  end-of-term  student  ratings  for  part-time 
faculty,  resulting  in  the  rejection  of  Hypothesis  2. 

Test  of  Hypothesis  3 

Hypothesis  3.   Part-time  faculty  assigned  to  both  the  mid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
higher  than  their  students'  ratings  of  instruction  at  mid- term  will 
lower  their  end-of-term  self-ratings  of  instruction. 

A  correlated  t-test  analysis  was  performed  to  test  the  hypothesis 
at  the  .05  level  of  significance  on  the  difference  scores  of  instructors 
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Table  6 


Results  of  the  Comparisons  of  Regression  Weights  of  Instructor  Over- 
raters  and  Underraters  in  the  Two  Feedback  Conditions 


UNDERRATERS 


OVERRATERS 


ITEM 


FEEDBACK 


BETA  WEIGHT 
(1-R) 


BETA  WEIGHT 
(1-R) 


Subject  Matter  Mastery 


21. 

Fact.  Knowl. 

yes 

-.038 

no 

.019 

22. 

Princ.  &  Theo. 

yes 

-.160 

no 

-.274 

24. 

Prof.  Skills  & 

yes 

.009 

Viewpoints 

no 

.002 

25 

Disc.  Methods 

yes 

.063 

no 

-.008 

Development  of  G 

eneral 

Skills 

23 

Think.  &  Prob. 

yes 

-.062 

Solving 

no 

-.058 

26 

Creative  Cap. 

yes 

-.083 

no 

-.280 

29 

Ef f ec .  Comm. 

yes 

-.015 

no 

-.209 

Personal  Development 

27 

Pers.  Respon. 

yes 

.095 

no 

-.294 

28 

Gen.  Lib.  Ed. 

yes 

-.094 

no 

.298 

30 

Implic.  for 

yes 

-.101 

Self  Undersg. 

no 

.205 

,585 


143 


.068 


-.756 


.027 


-1.188 


-.424 


-2.059* 


1.729* 


1.733* 


.018 
.044 


268 
332 


564 
,148 


.421 
.512 


263 
.064 


383 
112 


.097 
.229 


.090 
.712 


.206 
.080 


136 
.128 


-.203 


.996 


.739 


120 


746 


.649 


.992 


■1.796* 


1.071 


•1.861* 


*E<.05  one -tailed  test  for  differences  between  beta  weights 
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who  self -overrated  on  the  10  learning  objective  item  variables.   Again, 
since  instructor  self-ratings  were  not  computer  scored,  an  overall 
evaluation  T- score  was  not  generated  and  thus  was  not  available  for  use 
in  this  analysis.   As  indicated  in  Table  7,  results  showed  a  significant 
difference  on  9  of  the  10  learning  objective  item  means  in  the  predicted 
direction  of  the  hypothesis  (one-tailed  test  of  significance).   Although 
the  other  item  mean  was  not  found  significant  at  the  .05  level,  it  is 
worth  noting  that  the  direction  of  the  difference  was  in  the  predicted 
direction  of  the  hypothesis,  leading  the  researcher  to  conclude  that  the 
hypothesis  was  generally  supported.   Therefore,  part-time  faculty  who 
rated  themselves  higher  than  their  students  rated  them  at  mid- term 
(pretest)  tended  to  lower  their  end-of-term  (posttest)  self -ratings . 
This  suggested  that  instructor  self-rating  may  be  related  to  increased 
instructor  awareness  of  teaching  behavior. 

Test  of  Hypothesis  4 
Hypothesis  4.   Part-time  faculty  assigned  to  both  the  feedback  and 
no -feedback  groups  whose  self- ratings  of  instruction  are  equal  to  or 
lower  than  their  students'  ratings  of  instruction  at  mid- term  will  raise 
their  end-of-term  self-ratings  of  instruction. 

A  correlated  t-test  was  used  to  test  for  differences  (one-tailed 
test  of  significance).   No  overall  evaluation  T-score  was  generated  in 
self -rating,  and  therefore  was  not  available  for  this  analysis.   Six  out 
of  the  10  learning  objective  item  differences  tested  indicated  a  sig- 
nificance at  the  .05  level  (see  Table  7).   Although  not  found  sig- 
nificant, it  is  important  to  note  that  the  direction  of  the  other  four 
item  mean  differences  were  also  in  the  prediction  of  the  hypothesis.   In 
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Table  7 


Results  of  the  Comparisons  of  Change  Rates  in  Self-Ratlnps  from  Time  A 
to  Time  B  in  Instructor  Self -Overraters  and  Underraters 


ITEM 


OVERRATERS 
t-statistic    DIR. 


UNDERRATERS 
t-statistic    DIR. 


Subject  Matter  Mastery 

21.  Fact.  Knowl. 

22.  Princ.  &  Theo. 

24.  Prof.  Skills  & 
Viewpoints 

25.  Disc.  Methods    -1.75* 
Development  of  General  Skills 


-2.96* 

A>B 

2.82* 

B>A 

-3.74* 

A>B 

2.94* 

B>A 

-1.70* 

A>B 

1.61 

B>A 

A>B 


1.91* 


B>A 


23. 

Think.  &  Prob. 

-2.40* 

A>B 

1.09 

B>A 

Solving 

26. 

Creative  Cap. 

-.49 

A>B 

.43 

B>A 

29. 

Ef f ec .  Comm. 

-2.86* 

A>B 

2.14* 

B>A 

Personal  Development 

27 

Pers.  Respon. 

-2.37* 

A>B 

2.44* 

B>A 

28 

Gen.  Lib.  Ed. 

-3.53* 

A>B 

1.60 

B>A 

30 

Implic.for 

-3.07* 

A>B 

3.41* 

B>A 

Self  Undersg. 


*2<.05  one -tailed  test  for  differences  between  instructor  pretest  and 
posttest  self-ratings 
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other  words,  part-time  faculty  who  tended  to  rate  themselves  equal  to  or 
lower  than  their  students  rated  them  at  mid- term  (pretest)  rated  them- 
selves higher  on  the  end-of-term  (posttest)  ratings.   Therefore,  the 
findings  suggest  that  the  hypothesis  was  partially  supported. 

Because  of  the  findings  of  Hypotheses  3  and  4,  the  investigator 
continued  the  analysis  to  test  whether  the  significant  differences  found 
were  a  result  of  instructor  increased  awareness  of  his/her  own  teaching 
behaviors  or  representative  of  a  statistical  phenomenon  known  as  regres- 
sion to  the  mean.   It  is  common  in  some  research  areas  to  find  dif- 
ferences in  growth  rates  particularly  when  respondents  self- select  to 
receive  treatment.   The  point  in  this  test  is  that  many  patterns  of 
selection-maturation  should  lead  to  increased  within  group  variances  at 
the  end-of-term  (posttest)  when  compared  to  the  mid-term  (pretest). 

Table  8  shows  the  results  of  the  Anova  test  that  indicated  the  sig- 
nificant differences  of  within  group  variances  in  time  B  (posttest)  was 
found  for  both  overrater  and  underrater  instructor  groups.   Items  22, 
25,  and  23  were  found  significantly  different  at  the  .05  level  with 
respective  F  values  of  1.90,  3.41,  and  2.02  for  instructor  self- 
overraters.   Item  26  had  an  F  value  of  1.63  which  was  significant  at  the 
.05  level  for  the  self -underrater  group  of  instructors.   Therefore,  a 
similar  statistical  regression  for  both  groups  was  assumed,  and  it  was 
concluded  that  no  differential  statistical  regression  occurred.   In 
other  words,  the  differences  found  with  instructors  who  tended  to  self- 
overrate  or  underrate  appeared  to  be  a  result  of  instructor  increased 
awareness  of  his/her  own  teaching  behavior  rather  than  a  statistical 
regression  to  the  mean.   This  resulted  in  changed  final  (posttest)  in 
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Table  8 


Test  for  Differential  Statistical  Regression  Comparisons  of  Within  Group 
Variances  at  Times  A  and  B  for  Both  Instructor  Self-Overraters  and 
Underraters 


ITEM 


S2A 


S2B 


DIR. 


F- VALUE 


df 


Overraters 


A-B  21. 
A-B  22. 
A-B  24. 
A-B  25. 
A-B  23. 
A-B  26. 
A-B  29. 
A-B  27. 
A-B  28. 
A-B   30. 


.229 
.254 
.307 
.332 
.346 
.499 
.677 
.377 
.658 
.534 


.360 

B>A 

.483 

B>A 

1.188 

B>A 

1.133 

B>A 

.699 

B>A 

.519 

B>A 

1.256 

B>A 

.677 

B>A 

.862 

B>A 

.745 

B>A 

Underrate 

rs 

1.57 

1.90* 

3.88 

3.41* 

2.02* 

1.04 

1.85 

1.79 

1.31 

1.40 


33,35 
31,33 
25,24 
20,20 
30,30 
25,26 
28,28 
28,28 
20,21 
27,27 


A-B  21. 
A-B  22. 
A-B  24. 
A-B  25. 
A-B  23. 
A-B  26. 
A-B  29. 
A-B  27. 
A-B  28. 
A-B   30, 


.733 
.529 
.895 

1.137 
.644 
.937 

1.644 
.903 

1.401 

1.125 


.850 
.752 
1.172 
1.795 
.997 
1.531 
1.972 
1.187 
1.768 
1.655 


B>A 
B>A 
B>A 
B>A 
B>A 
B>A 
B>A 
B>A 
B>A 
B>A 


1.16 

1.42 

1.31 

1.58 

1.55 

1.63* 

1.19 

1.31 

1.26 

1.47 


57,31 
60,36 
67,42 
69,43 
58,34 
66,41 
61,37 
61,38 
70,46 
63,39 


*£<.05 
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structor  self-ratings  which  would  more  closely  approximate  the  "reality" 
of  student  ratings  of  instruction. 

Test  for  Hypothesis  5 

Hypothesis  5.   Part-time  faculty  teaching  long-term  courses  who 
receive  mid- term  student  ratings  feedback  will  receive  higher  end-of- 
term  student  ratings  than  part-time  faculty  teaching  short-term  courses 
who  receive  mid- term  student  ratings  feedback. 

Results  of  the  analyses  of  variance  performed  on  the  combined  10 
learning  objective  item  means  and  the  overall  evaluation  T- score  of  the 
two  feedback  groups  (groups  1  and  2)  between  instructor  levels  (long- 
term  courses  versus  short-term  courses)  revealed  no  significant  dif- 
ferences at  the  .05  level  in  the  direction  predicted  by  the  hypothesis 
(see  Table  9).   As  indicated  in  the  Anova  summary  table  (see  Appendix 
P) ,  there  were  fairly  large  F  values  for  four  item  means  that  would  have 
been  significant  if  the  prediction  of  the  hypothesis  had  been  in  the  op- 
posite direction. 

Results  of  the  test  of  Hypothesis  5  seemed  to  indicate  that  the 
hypothesis  was  not  supported  in  the  predicted  direction,  and  therefore, 
the  hypothesis  was  rejected.   Nine  of  the  11  item  means  showed  higher 
end-of-term  student  ratings  for  part-time  faculty  teaching  short-term, 
non-credit  courses  than  for  part-time  faculty  teaching  long-term,  credit 
courses.   Findings  suggest  that  the  feedback  condition  may  have  had  sig- 
nificant effects  on  subsequent  student  ratings  of  part-time  instructors 
of  short-term  rather  than  long-term  courses. 
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Table  9 

Results  of  the  Comparisons  of  the  Means  and  Standard  Deviations  of 
Short-Term  Course  Versus  Long-Term  Course  Instructors  in  the  Feedback 
Condition 


ITEM 


SHORT-TERM 

(n-53) 
MEAN   S . D . 


LONG-TERM 

(n-41) 

MEAN    S . D . 


Subject  Matter  Mastery 

21.  Fact.  Know.      4.10  .352 

22.  Princ.  &  Theo.    4.06  .346 

23.  Prof.  Skills  &   3.95  .571 
Viewpoints 

25.  Disc.  Methods    3.65  .720 

Development  of  General  Skills 

23.  Think.  &  Prob.    4.05  .431 
Solving 

26.  Creative  Cap.     3.76  .662 
29.  Effec.  Comm.     3.42  .955 

Personal  Development 


3.69  .590 

3.73  .581 

3.60  .602 

3.40  .618 


DIR. 


3.74 

.564 

S>L 

3.82 

.478 

S>L 

3.65 

.520 

S>L 

L>S 

S>L 

S>L 
S>L 


27. 

Pers.  Respon. 

3.89 

.652 

3.66 

.499 

S>L 

28. 

Gen.  Lib.  Ed. 

3.38 

.831 

3.13 

.704 

S>L 

30. 

Implic.  for 
Self  Undersg. 

3.83 

.604 

3.50 

.530 

S>L 

Overall  score 

3.19 

1.003 

3.60 

1.191 

L>S 

67 
Test  of  Hypothesis  6 

Hypothesis  6.   There  is  no  significant  difference  between  the 
opinions  of  part-time  faculty  teaching  long-term  and  part-time  faculty 
teaching  short-term  courses  towards  student  ratings  of  instruction. 

A  comparison  of  the  responses  of  the  instructors  of  short-term 
courses  and  instructors  of  long-term  courses  to  the  five  items  on  the 
Instructor  Questionnaire  was  made  utilizing  a  t- statistic.   The  ques- 
tionnaire, developed  by  the  researcher,  was  designed  to  assess  the 
opinions  of  part-time  instructors  towards  student  ratings  of  instruc- 
tion.  The  first  two  items,  1  and  2,  were  more  informational  in  nature 
and  measured  how  much  importance  instructors  felt  their  respective  in- 
stitution currently  placed  and  should  place  on  student  ratings  evalua- 
tions.  The  other  three  items,  3,  4,  and  5,  were  used  to  ask  the  in- 
structor how  important  he/she  personally  felt  student  ratings  of  in 
struction  were,  and  therefore,  measured  instructor  opinions  towards  stu- 
dent ratings. 

As  presented  in  Table  10,  the  results  indicated  statistically 
significant  differences  at  the  .05  level  (two-tailed  test)  on  two  of  the 
three  items  measuring  instructor  opinions  (items  3  and  5).   Item  3  had  a 
t- statistic  of  2.20  and  asked  the  instructor  how  much  weight  should  be 
given  by  student  evaluation  of  instruction  to  his/her  overall  evaluation 
as  an  instructor.   The  direction  of  the  difference  showed  that  more  in- 
structors of  short-term  courses  weighted  this  item  more  importantly  than 
did  instructors  of  long-term  courses.   Item  5  of  the  questionnaire  asked 
instructors  how  much  importance  he/she  placed  on  knowing  how  satisfied 
students  were  with  his/her  teaching  and  had  a  t-statistic  of  2.76. 
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Table  10 


Results  of  the  Comparisons  of  the  Means  and  Standard  Deviations  of  the 
Instructor  Questionnaire  (Lonp-Term  Versus  Short-Term) 


ITEM 

SHORT 

-TERM 

LONG- 

TERM 

DIR. 

(n= 

53) 

(n= 

=41) 

MEAN 

S.D. 

MEAN 

S.D. 

1.  Import,  to  Org. 

3.57 

1.140 

3.48 

1.040 

S>L 

2.  Import.  Should 

3.84 

.857 

3.81 

.943 

S>L 

3.  Weight  in  Eval . 

3.96 

.790 

* 

3.54 

1.030 

S>L 

4.  Usefulness  of 

3.96 

.970 

3.60 

1.110 

S>L 

Results 

5.  Import,  to 

4.81 

.445 

* 

4.43 

.860 

S>L 

Teaching 

*  £<.05  two-tailed  test 


Again,  instructors  of  short-term  courses  rated  this  item  of  more  impor- 
tance than  instructors  of  long-term  courses.   The  other  item  means 
(items  1,  2,  and  4),  although  not  significant,  indicated  instructors  of 
short-term  courses  rated  these  items  higher  than  did  the  instructors  of 
long-term  courses.   Based  on  the  above  mentioned  findings,  the  results 
seemed  to  suggest  that  Hypothesis  6  be  rejected.   Therefore,  part-time 
instructors  teaching  short-term,  non- credit  courses  seemed  to  have 
valued  student  ratings  of  instruction  more  than  instructors  of  long- 
term,  credit  courses. 

Post  Hoc  Analyses 
A  one-way  analysis  of  variance  was  used  on  the  end-of-term 
(posttest)  student  ratings  of  the  10  learning  objective  item  means  and 
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the  overall  evaluation  T- score  of  the  IDEA  standard  form  to  analyze  the 
instructor  demographic  information  regarding  sex,  educational  degree  ob- 
tained, and  teaching  experience.   In  the  analysis  of  the  instructor  sex 
variable,  only  one  item  was  found  significant  at  the  .05  level.   On  this 
item  (item  27,  personal  responsibility),  female  instructors  received  a 
higher  end-of-term  rating  than  did  male  instructors.   The  remaining  10 
item  variables  analyzed  showed  no  significant  differences  between  the 
student  ratings  of  male  and  female  instructors.   The  one-way  Anova  per- 
formed on  the  10  learning  objective  item  means  and  the  overall  evalua- 
tion T- score  regarding  the  instructor  variable  of  educational  degree  ob- 
tained revealed  no  significant  differences  among  the  three  educational 
groups:   bachelor's  degree,  master's  degree,  and  doctorate.   The  Anova 
applied  to  the  end-of-term  (posttest)  student  ratings  of  the  10  learning 
objective  item  means  and  the  overall  evaluation  T- score  regarding  teach- 
ing experience  showed  significant  differences  at  the  .05  level  on  6  of 
the  10  learning  objective  item  means  (items  21,  22,  23,  26,  27,  and  30). 
The  overall  T-score  was  also  significant  at  the  .05  level  (see  Table 
11) .   These  significant  differences  were  nearly  equally  distributed 
among  the  three  category  groups  of  items  (subject  matter  mastery, 
general  skills,  and  personal  development).   Follow-up  analysis  for  those 
items  indicating  a  signficant  F  value  was  conducted  utilizing  Tukey's 
method  to  test  for  between  group  differences.   Overall,  Tukey's  method 
revealed  that  students  rated  instructors  with  the  least  amount  of  ex- 
perience, group  1  (0-3  years),  lower  on  each  of  the  six  significant  item 
variables  than  the  other  three  teaching  experience  groups,  [group  2  (4-9 
years),  group  3  (10-14  years),  and  group  4  (15+  years)]  at  the  .05 


70 


Table  11 


Tukev's  Test  for  Between  Group  Differences  on  Instructor 
Experience  Variable 


ITEM 


F -VALUE      df 


TUKEY'S  DIFF. 
BY  GROUP 


Subject  Matter  Mastery 

21.  Fact.  Know. 

22.  Princ.  &  Theo. 

24.  Prof.  Skills  & 
Viewpts . 

25.  Disc.  Methods 


6.51** 

3,89 

1<   2,3,4 

8.71** 

3,89 

1<   2,3,4 

2.45 

3,89 

1.33 


Development  of  General  Skills 

23.  Think  &  Prob. 
Solving 


26.  Creative  Cap. 
29.  Effec.  Coram. 

Personal  Development 

27.  Pars.  Respon. 

28.  Gen.  Lib.  Ed. 


30.  Implic.  for 
Self  Undersg. 

Overall 


3,89 


7.68** 

3,89 

1<  2,3,4 

4 . 14** 

3,89 

1,2<  3,4 

1.73 

3,89 

5.51** 

3,89 

2.30 

3,89 

4.95** 

3,89 

2.78* 


3,89 


1,3<  2,4 

1<  2,3,4 
1<  2,3,4 


*  E<.05 


**  £<.01 
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level.   Therefore,  comparisons  of  part-time  instructors  across  levels  on 
sex  and  educational  degree  attainment  variables  showed  similarities  on 
student  ratings  that  were  greater  than  the  differences.   The  one  major 
exception  was  the  instructor  variable  related  to  teaching  experience. 
The  part-time  instructors  who  reported  three  years  experience  or  less 
were  rated  significantly  lower  than  instructors  with  more  experience. 
Tukey's  results  indicated  that  the  more  teaching  experience  the  instruc- 
tor claimed,  the  higher  the  end- of- term  student  ratings.   Therefore,  of 
the  three  instructor  demographic  variables  investigated,  amount  of 
teaching  experience  seemed  to  be  significantly  related  to  student 
ratings . 

One  other  additional  analysis  performed  by  the  researcher  was  re- 
lated to  differences  between  student  motivation  levels  (short-term 
course  students  versus  long-term  course  students).   The  IDEA  standard 
form  computer- generated  summary  report  utilized  in  the  study  reported  a 
class  motivation  level  score  from  I-V  with  I-lowest  and  V-highest.   This 
class  motivation  level  score  was  based  on  the  mean  student  ratings  to 
item  36  of  the  evaluation  form.   This  item  asked  students  to  rate  them- 
selves on  a  1-5  basis  on  how  strong  their  desire  was  to  take  this  par- 
ticular course.   As  the  researcher  consulted  with  the  instructors  as- 
signed to  the  full -feedback  group,  she  noticed  that  there  appeared  to  be 
distinct  differences  between  the  class  motivation  levels  printed  on  the 
summary  reports  of  instructors  teaching  short-term  courses  and  instruc- 
tors teaching  long-term  courses.   Therefore,  a  post  hoc  analysis  was 
conducted  to  test  this  perceived  difference.   A  one-way  analysis  of 

was  performed  to  test  for  differences  between  motivation  levels 


variance 
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reported  by  students  enrolled  in  short-term  courses  and  students  en- 
rolled in  long-term  courses  at  the  mid-term  (pretest)  and  end-of-term 
(posttest)  evaluation  points.   As  indicated  in  Table  12,  findings 
clearly  showed  a  significant  difference  between  motivation  levels  of 
students  who  enroll  in  short-term,  non-credit  courses  and  students  who 
enroll  in  long-term,  credit  courses  at  the  .05  level,  with  the  short- 
term  course  students  being  more  highly  motivated. 

Summary  of  the  Chapter 
This  chapter  contains  the  findings  of  the  study.   A  total  of  94  in- 
structors participated  in  the  study  and  received  student  ratings  of  in- 
struction and  self- ratings  of  instruction  summary  reports  which  were 
analyzed.   In  addition,  94  Instructor  Questionnaires  developed  by  the 
investigator  were  completed  and  analyzed.   Each  of  the  six  original 
hypotheses  were  tested.   Of  the  six  hypotheses  analyzed,  five  were 
tested  in  reference  to  one  of  the  two  dependent  variables  in  the  study, 
either  end-of-term  student  ratings  of  instruction  or  end-of-term  in- 
structor self-ratings  utilizing  the  10  learning  objective  means  of  the 
IDEA  standard  form  and  the  overall  evaluation  T- score.  The  sixth 

Table  12 


RATINGS 

SHORT-TERM 
MEAN 

LONG-TERM 
MEAN 

F- VALUE 

df 

Mid- term 
End-of-term 

4.30 

4.42 

2.51 
2.45 

53.80** 
80.52** 

1,68 
1,92 

*  £<.05    **  E<.01 
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the  overall  evaluation  T-score.   The  sixth  hypothesis  utilized  the  last 
three  items  of  the  five -item  Instructor  Questionnaire  developed  by  the 
investigator  to  analyze  for  differences  in  instructor  attitudes  toward 
student  ratings  of  instruction.   Post  hoc  analyses  utilized  additional 
Anovas  to  investigate  the  differences  between  the  instructor  demographic 
variables  and  student  motivation  levels. 

An  Anova  was  utilized  to  test  for  overall  differences  on  the  11 
student  rating  item  means  between  the  feedback  and  no- feedback  treatment 
groups.   Follow-up  analyses  were  done  utilizing  Fisher's  LSD  method  for 
multiple  comparisons  of  each  treatment  group  to  test  for  specific  dif- 
ferences between  the  significantly  different  item  means  and  to  test  any 
sensitizing  effects  the  pretest  may  have  had  on  posttest  student  ratings 
and  instructor  self -ratings .   Hypothesis  1  analyzed  the  effect  the  full 
feedback  condition  (printed  feedback  with  oral  consultation)  and  the 
partial  feedback  condition  (printed  feedback  only)  had  on  subsequent 
student  ratings  of  part-time  faculty  instruction.   The  overall  Anova  in- 
dicated only  5  out  of  11  item  means  significantly  different  at  the  .05 
level.   Of  the  30  paired  item  analyses  performed,  only  15  learning  ob- 
jective item  means  were  found  significant  at  the  .05  level,  resulting  in 
a  rejection  of  Hypothesis  1.   A  related  samples  t-test  of  paired  com- 
parisons testing  for  differences  between  the  mid-term  and  end-of-term 
ratings  means  on  each  of  the  11  items  within  the  two  feedback  and  one 
no- feedback  groups  showed  the  greatest  amount  of  change  in  student 
ratings  occurred  within  the  full  feedback  group.   Although  the 
hypothesis  was  rejected,  overall  results  indicated  that  mid-term  student 
ratings  feedback  seemed  to  have  had  a  moderate,  but  non-signif leant ,  ef- 
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feet  on  subsequent  student  ratings  of  part-time  faculty.   There  were  no 
significant  differences  found  between  the  two  feedback  conditions,  in- 
dicating that  the  mid- term  printed  feedback  condition  may  be  at  least  as 
effective  as  mid- term  printed  feedback  with  oral  consultation  on  sub- 
sequent student  ratings  of  part-time  faculty. 

Hypothesis  2  utilized  a  regression  analysis  model  to  test  whether 
instructor  self -overrating  was  a  valid  predictor  of  end-of-term  student 
ratings  for  part-time  faculty.   Of  the  20  t-tests  comparing  differences 
in  regression  weights  of  instructor  self -overraters  and  underraters, 
none  of  the  overraters'  regression  weights  were  significant  at  the  .05 
level.   The  results  indicated  that  overrating  was  not  a  valid  predictor 
of  end-of-term  student  ratings,  and  the  hypothesis  was,  therefore, 
rejected. 

Hypotheses  3  and  4  tested  the  relationship  of  instructor  self- 
overrating  and  self -underrating  to  end-of-term  self -ratings .   Of  the  20 
correlated  t-tests  performed  to  test  each  hypothesis  on  the  10  learning 
objective  item  variables,  15  were  significant  at  the  .05  level;  9  item 
means  for  the  self -overraters  and  6  item  means  for  the  self -underraters , 
thus  supporting  Hypothesis  3  and  paritally  supporting  Hypothesis  4. 
Hypothesis  5  tested  for  differences  of  the  end-of-term  student 
ratings  between  instructors  of  short-term  courses  and  instructors  of 
long-term  courses  in  the  feedback  condition.   Of  the  10  learning  objec- 
tive items  and  the  overall  evaluation  T-score  tested,  no  significant 
differences  were  found  at  the  .05  level  in  the  direction  predicted  by 
the  hypothesis.   Nine  of  the  11  item  variable  means,  however,  showed 
higher  end-of-term  student  ratings  for  part-time  faculty  teaching  short- 
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term,  non-credit  courses  than  for  part-time  faculty  teaching  long-term, 
credit  courses.   Findings  suggested  that  the  hypothesis  be  rejected, 
since  results  seemed  to  indicate  that  mid-term  ratings  feedback  may  have 
had  moderate  effects  on  the  subsequent  student  ratings  of  part-time  in- 
structors teaching  short-term,  non-credit  courses. 

Hypothesis  6  utilized  a  t-statistic  to  test  for  differences  between 
instructor  levels  (short-term  versus  long-term)  on  instructor  ratings  of 
five  items  on  the  Instructor  Questionnaire  developed  by  the  researcher 
to  assess  instructors'  opinions  towards  student  ratings  of  instruction. 
Of  the  five  t-statistic  analyses  performed,  results  indicated  that  dif- 
ferences in  responses  to  two  of  the  three  items  measuring  instructor 
opinions  towards  student  ratings  were  significant  at  the  .05  level  with 
the  direction  of  the  difference  indicating  a  higher  ratings  mean  for  in- 
structors of  short-term  courses.   Although  found  non-significant  at  the 
.05  level,  the  other  three  items  also  showed  a  higher  ratings  mean  from 
instructors  of  short-term  courses.   Therefore,  the  null  hypothesis  was 
rejected,  indicating  that  part-time  instructors  teaching  short-terra 
courses  seemed  to  value  student  ratings  of  instruction  more  than  part- 
time  instructors  teaching  long-term  courses. 

Post  hoc  analyses  of  variance  were  utilized  to  test  for  differences 
on  instructor  demographic  characteristics.   Results  indicated  that  of 
the  three  instructor  demographic  variables  tested,  only  one,  teaching 
experience,  seemed  to  be  significantly  related  to  the  dependent  vari- 
able, student  ratings  of  instruction,  at  the  .05  level  of  significance. 

Additional  analyses  of  variance  were  performed  to  test  for  dif- 
ferences in  class  motivation  levels  comparing  the  means  of  item  36  of 
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the  IDEA  standard  form.   Findings  clearly  indicated  students  of  short- 
term,  non- credit  courses  were  significantly  more  motivated  than  students 
of  long-term,  credit  courses  at  the  .05  level. 

Considering  all  variables  of  interest  tested,  the  similarities  be- 
tween the  two  feedback  treatment  groups  were  greater  than  the  dif- 
ferences across  instructor  levels,  indicating  that  printed  mid- term 
feedback  may  be  at  least  as  effective  as  printed  mid- term  feedback  with 
oral  consultation  on  subsequent  student  ratings  of  part-time  faculty. 
Paired  within  group  comparisons  of  the  feedback  and  no -feedback  condi- 
tions, however,  showed  greater  significant  differences  in  the  change  of 
subsequent  student  ratings  for  the  full  feedback  group  (printed  feedback 
with  oral  consultation)  than  for  the  partial  feedback  group  (printed 
feedback  only).   Further  investigation  between  instructor  levels  (short- 
term  course  versus  long-term  course)  indicated  an  opposite  effect  than 
predicted.   Results,  therefore,  seemed  to  suggest  that,  overall,  mid- 
term feedback  had  a  greater  moderate  significant  effect  (.05  level)  on 
the  subsequent  end-of-term  student  ratings  of  part-time  instructors  of 
short-term,  non-credit  courses  than  on  the  subsequent  student  ratings  of 
instructors  of  long-term,  credit  courses. 


CHAPTER  FIVE 
SUMMARY,  CONCLUSIONS,  IMPLICATIONS,  AND  RECOMMENDATIONS 

This  chapter  presents  a  summary  of  the  study,  conclusions,  implica- 
tions, and  recommendations  related  to  the  findings. 

Summary 

The  primary  purpose  of  this  study  was  to  determine  the  effect  dif- 
ferent forms  of  student  ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .   Part-time  faculty 
opinions  towards  student  ratings  of  instruction  also  were  explored.   The 
Instructional  Development  and  Effectiveness  Assessment  (IDEA)  standard 
form  (Kansas  State  University,  1975)  was  the  evaluation  tool  utilized  in 
the  study  to  assess  the  two  dependent  variables,  student  ratings  of  in- 
struction and  instructor  self -ratings .   The  researcher  developed  the  In- 
structor Questionnaire  (see  Appendix  D)  to  measure  the  degree  to  which 
an  instructor  felt  the  institution  valued  student  ratings  of  instruction 
and  how  much  an  instructor  personally  valued  student  ratings  of  instruc- 
tion.  Prior  to  the  beginning  of  the  study,  the  researcher  gathered  in- 
structor demographic  information  regarding  sex,  educational  degree  ob- 
tained, and  years  of  teaching  experience. 

A  pilot  study  consisting  of  eight  part-time  instructors  teaching 
short-term  courses  was  conducted  to  assess  the  feasibility  of  the  two 
instruments.   Student  ratings  of  instruction  data,  instructor  self- 
ratings  data,  and  response  data  from  the  Instructor  Questionnaire  were 
gathered  and  verified.   The  final  study  included  a  total  of  94  part-time 
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faculty  from  two  different  post-secondary  institutions.   Fifty- three  of 
the  participants  were  instructors  of  short-term,  non-credit  courses  from 
the  Division  of  Lifelong  Learning  of  the  School  of  Extended  Studies  at 
the  University  of  South  Florida  and  41  were  part-time  instructors  of 
long-term,  credit  courses  from  the  Educational  Services  Department  of 
the  Weekend  College  Division  at  Saint  Leo  College. 

A  one-way  analysis  of  variance  was  used  to  test  Hypothesis  1  on  the 
10  learning  objective  item  means  and  the  overall  evaluation  T- score. 
Follow-up  analysis  for  those  items  indicating  a  significant  F  value  was 
conducted  utilizing  Fisher's  LSD  method  for  multiple  comparisons  to  fur- 
ther analyze  for  between  treatment  group  differences.   In  total,  30 
Fisher's  LSD  tests  for  paired  groups  differences  were  performed  to  test 
Hypothesis  1.   In  each  of  these  analyses,  the  dependent  variable  was 
student  ratings  of  instruction.   Further  Anova  analyses  to  test  for  the 
sensitizing  effect  of  the  pretest  treatment  on  the  posttest  (end-of- 
term)  student  ratings  of  instruction  and  instructor  self-ratings  showed 
no  significant  effects  at  the  .05  level. 

Hypothesis  1.   Part-time  faculty  who  receive  printed  summary  feed- 
back along  with  oral  consultation  about  their  mid- term  student  ratings 
will  receive  higher  end-of-term  student  ratings  than  part-time  faculty 
who  receive  only  printed  summary  feedback  of  mid-term  student  ratings, 
and  part-time  faculty  who  receive  only  printed  summary  feedback  of  mid- 
term student  ratings  will  receive  higher  end-of-term  student  ratings 
than  part-time  faculty  who  do  not  receive  mid- term  student  ratings  feed- 
back. 
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Hypothesis  1  was  not  supported  at  the  .05  level  of  significance  in 
a  general  analysis  of  variance  test  for  differences  on  each  of  the  11 
item  means.   However,  Fisher's  LSD  paired  comparison  tests  for  specific 
differences  between  treatment  groups  on  the  5  items,  that  had  sig- 
nificant differences  in  the  general  Anova  showed  15  comparisons  were 
significant  at  the  .05  level  out  of  a  possible  30  comparisons.   This 
finding  resulted  in  a  rejection  of  Hypothesis  1,  although  inspection  of 
the  means  of  the  10  item  learning  objectives  and  the  overall  evaluation 
T-score  of  the  two  feedback  conditions  were  in  the  predicted  direction 
indicating  a  positive,  but  non-significant  effect  for  the  instructors 
who  received  the  full  feedback  (printed  feedback  with  oral  consultation) 
and  partial  feedback  (printed  feedback  only) .   The  instructors  who 
received  full  feedback  of  mid- term  student  ratings  (printed  feedback 
with  oral  consultation)  did  not  appear  to  receive  significantly  higher 
end- of- term  student  ratings  than  instructors  who  received  partial  feed- 
back (printed  feedback  only).   Generally,  part-time  instructors  who 
received  printed  feedback  of  mid- term  student  ratings  tended  to  receive 
higher  end- of- term  student  ratings  on  more  learning  objective  item  vari- 
ables and  the  overall  evaluation  T-score  than  did  instructors  who 
received  mid- term  printed  feedback  with  oral  consultation  or  who 
received  no  mid- term  printed  feedback  of  student  ratings.   This  finding 
suggests  that  mid- term  printed  feedback  may  be  at  least  as  effective  on 
subsequent  student  ratings  of  part-time  faculty  as  mid- term  printed 
feedback  with  oral  consultation.   Within  group  paired  comparisons, 
however,  seemed  to  indicate  that  the  full  feedback  group  (printed  feed- 
back with  oral  consultation)  showed  significant  change  on  more  student 


80 

ratings  item  means  from  the  mid- term  evaluation  to  the  end- of -term 
evaluation  than  did  the  other  three  treatment  groups. 

Hypotheses  2,  3,  and  4  tested  the  influence  of  instructor  self- 
ratings  on  the  10  learning  objective  item  means.   No  overall  evaluation 
score  was  generated  from  instructor  self- ratings ,  and  thus  was  not 
analyzed. 

Hypothesis  2.   Part-time  faculty  assigned  to  both  the  mid-term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
higher  than  their  students'  ratings  of  instruction  at  mid- term  will 
receive  higher  end- of- term  student  ratings  than  part-time  faculty  whose 
self- ratings  of  instruction  are  equal  to  or  lower  than  their  students' 
ratings  of  instruction  at  mid- term. 

Hypothesis  2  utilized  a  linear  regression  statistical  model  to  cal- 
culate regression  weights  and  a  t-statistic  to  test  the  hypothesis. 
Based  on  the  equilibrium  theory  used  in  Centra's  (1973b)  study,  this 
hypothesis  tested  instructor  self -overrating  in  the  feedback  condition 
as  a  predictor  of  end-of-term  student  ratings.   Regression  weights  were 
calculated  for  the  10  learning  objective  item  means  in  two  instructor 
categories:   those  who  self -overrated  and  instructors  who  self- 
underrated  for  both  the  feedback  and  no-feedback  conditions.   The  40 
regression  weights  were  then  tested  with  a  one-tailed  t-statistic  to 
analyze  for  differences  between  the  feedback  and  no- feedback  conditions 
for  each  instructor  category:   underraters  and  overraters .   The  t-test 
results  indicated  differences  on  only  5  out  of  20  regression  weights 
analyzed  were  significant  at  the  .05  level.   The  results,  therefore,  in- 
dicated that  the  hypothesis  was  not  supported. 


81 
Hypothesis  3.   Part-time  faculty  assigned  to  both  the  mid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
higher  than  their  students'  ratings  of  instruction  will  change  their 
end-of-term  self-ratings  of  instruction. 

A  one-tailed  correlated  t-test  statistic  was  performed  to  test  for 
differences  on  the  10  learning  objective  items.   Results  indicated  that 
the  differences  in  all  10  item  means  were  in  the  predicted  direction  of 
the  hypothesis  and  9  out  of  10  differences  in  item  means  were  sig- 
nificant at  the  .05  level. 

This  seemed  to  indicate  that  instructors  who  both  rated  their  own 
evaluation  of  instruction  higher  than  their  actual  student  ratings  and 
who  received  student  ratings  feedback  tended  to  lower  their  subsequent 
self- ratings  score.   The  hypothesis  was,  therefore,  generally  supported. 

Hypothesis  4.   Part-time  faculty  assigned  to  both  the  raid- term 
feedback  and  no-feedback  groups  whose  self-ratings  of  instruction  are 
equal  to  or  lower  than  their  students'  ratings  of  instruction  at  mid- 
term will  raise  their  end-of-term  self-ratings  of  instruction. 

One-tailed  correlated  t-test  results  indicated  that  the  differences 
in  only  6  out  of  10  learning  objective  items  significant  at  the  .05 
level,  resulting  in  a  partial  acceptance  of  Hypothesis  4.   Since  the  in- 
structors in  Hypothesis  3  who  tended  to  overrate  themselves  at  mid- term 
significantly  changed  their  end-of-term  self-ratings  by  lowering  their 
scores  on  9  of  the  10  learning  objective  item  variables,  and  the  in- 
structors in  Hypothesis  4  who  tended  to  underrate  themselves  at  mid- term 
significantly  changed  their  end-of-term  self-ratings  by  increasing  their 
scores  on  6  of  the  10  learning  objective  item  variables,  an  additional 
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t-test  statistic  was  performed  to  test  for  the  statistical  phenomenon 
known  as  regression  to  the  mean.   Results  of  this  analysis  indicated  no 
differential  statistical  regression.   Therefore,  results  confirmed  sup- 
port for  Hypothesis  3  and  partial  support  for  Hypothesis  4. 

Hypothesis  5.   Part-time  faculty  teaching  long-term  courses  who 
receive  mid- term  student  ratings  feedback  will  receive  higher  end-of- 
term  student  ratings  than  part-time  faculty  teaching  short-term  courses 
who  receive  mid-term  student  ratings  feedback. 

Results  of  the  Anova  performed  on  the  10  learning  objective  item 
means  and  the  overall  evaluation  T-score  indicated  no  significant  dif- 
ferences in  the  direction  predicted  by  the  hypothesis  at  the  .05  level 
between  the  instructors  of  long-term  courses  and  instructors  of  short- 
term  courses  in  the  feedback  condition.   Inspection  of  the  end-of-term 
(posttest)  student  ratings  means,  however,  indicated  that  9  of  the  11 
item  means  were  higher  for  part-time  instructors  teaching  short-term, 
non-credit  courses  than  for  part-time  instructors  teaching  long-term, 
credit  courses.   Results  seemed  to  indicate  that  mid- term  student 
ratings  feedback  tended  to  have  had  a  more  positive  effect  on  the  end- 
of-term  student  ratings  of  short-term  course  instructors;  these  results 
were  the  reverse  of  the  prediction.   Therefore,  based  on  the  statistical 
analyses,  Hypothesis  5  was  rejected. 

Hypothesis  6.   There  is  no  significant  difference  between  the 
opinions  of  part-time  faculty  teaching  both  long-term  and  short-term 
courses  towards  student  ratings  of  instruction. 

Hypothesis  6  tested  for  differences  in  instructor  opinions  towards 
student  ratings  of  instruction  utilizing  a  t-statistic  analysis  on  the 
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five  items  of  the  Instructor  Questionnaire.   No  significant  differences 
were  found  at  the  .05  level  between  instructors  on  the  first  two  items 
of  the  Instructor  Questionnaire.   These  items,  however,  were  more  infor- 
mational in  nature  and  measured  how  much  importance  instructors  felt 
their  respective  institutions  placed  and  should  place  on  student 
opinions  of  instruction.   The  results  of  these  two  items,  although  not 
significant  at  the  .05  level,  indicated  higher  means  for  instructors  of 
short-term  courses.   The  other  three  items  measured  instructor  opinions 
towards  student  ratings  of  instruction.   The  t-test  results  indicated 
that  the  differences  in  two  of  these  three  items  were  significant  at  the 
.05  level  with  instructors  of  short-term  courses  placing  higher  value  on 
student  ratings  of  instruction  than  the  instructors  teaching  long-term 
courses.   The  other  item,  although  found  non-significant  at  the  .05 
level,  also  indicated  a  higher  mean  for  instructors  of  short-term 
courses.   The  findings  seemed  to  suggest  that  part-time  faculty  teaching 
short-term,  non-credit  courses  valued  student  ratings  of  instruction 
more  than  instructors  teaching  long-term,  credit  courses. 

Additional  analyses  were  made  on  three  instructor  demographic  vari- 
ables and  on  class  motivation  levels.   The  instructor  demographic  data 
of  sex,  educational  degree  obtained,  and  teaching  experience  were  tested 
with  an  Anova  on  the  10  learning  objective  items  and  the  overall  evalua- 
tion T-score.   Results  showed  only  teaching  experience  as  a  critical 
variable  in  relation  to  subsequent  student  ratings  of  instruction.   The 
instructors  with  the  least  amount  of  teaching  experience  (0-3  years) 
were  rated  significantly  lower  on  6  of  the  10  learning  objective  item 
means  and  the  overall  evaluation  T-score  at  the  .05  level. 
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Student  motivation  levels  were  analyzed  with  a  one-way  Anova  on  the 
mid- term  and  end-of-term  ratings  of  the  class  motivation  level  score 
reported  on  the  IDEA  standard  form  printed  summary  report.   Results  in- 
dicated clear  significant  differences  at  the  .05  level  between  students 
enrolled  in  long-term  courses  and  students  enrolled  in  short-term 
courses  on  both  the  mid- term  and  end-of-term  ratings.   Students  enrolled 
in  short- terra  courses  were  more  highly  motivated  than  students  enrolled 
in  long-term  courses. 

When  considering  the  results  of  the  study,  certain  limitations  in 
regard  to  their  generalizability  should  be  kept  in  mind.   One  limitation 
was  the  issue  of  instructor  volunteers.   Even  though  only  two  instruc- 
tors of  long-term  courses  and  one  instructor  of  short-term  courses  chose 
not  to  participate  in  the  study,  there  is  the  possibility  that  these 
non-participating  instructors  differed  from  the  ones  that  participated. 
In  addition,  the  quasi-experimental  design  of  the  study  did  not  allow 
random  assignment  of  students  to  treatment  groups.   This  lack  of  random 
assignment  could  have  resulted  in  very  different  or  biased  groups  per 
instructor. 

Another  limitation  was  that  the  study  sample  was  drawn  from  the 
part-time  instructor  population  of  one  state  university's  non-credit 
course  program  and  one  private  college's  non- traditional  weekend  college 
credit  course  program.   Since  the  study  involved  only  two  institutions, 
generalizations  to  other  institutions  would  be  restricted.   A  final 
limitation  was  the  fact  that  the  researcher  and  the  course  coordinator 
were  the  same  person  functioning  as  the  instructors'  source  of  feedback 
information.   This  may  have  resulted  in  experimenter  bias. 
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Conclusions 

The  results  of  analyses  of  variance  of  Hypothesis  1  regarding  the 
first  dependent  variable,  student  ratings  of  instruction,  indicated  al- 
though non-significant  overall,  the  instructors  assigned  to  the  two  stu- 
dent ratings  feedback  conditions  appeared  to  have  improved  on  subsequent 
student  ratings.   Closer  analysis  of  differences  between  the  two  feed- 
back conditions  showed  that  the  similarities  were  greater  than  the  dif- 
ferences between  the  two  experimental  groups,  suggesting  that  the  mid- 
term printed  feedback  may  have  been  at  least  as  effective  as  mid- term 
printed  feedback  with  oral  consultation.   However,  it  is  important  to 
note  that  although  the  full  feedback  group  (printed  feedback  with  oral 
consultation)  was  not  statistically  different  from  the  printed  feedback 
only  group,  results  suggest  that  mid- term  printed  feedback  with  oral 
consultation  of  student  ratings  did  minimally  effect  some  changes  in 
subsequent  student  ratings  of  part-time  faculty  instructional  behavior. 
Findings  also  suggest  that  the  full  feedback  condition  (printed  feedback 
with  oral  consultation)  showed  improved  end-of-term  student  ratings  of 
the  instructors  who  received  that  type  of  feedback.   In  other  words,  in- 
structors who  received  printed  summary  feedback  with  oral  consultation 
did  receive  better  end-of-term  student  ratings  on  certain  item  vari- 
ables . 

Results  of  the  analysis  of  the  effect  of  feedback  between  instruc- 
tor levels  (instructors  of  short-term  courses  versus  instructors  of 
long-term  courses)  led  the  researcher  to  conclude  that  student  ratings 
feedback  appears  to  have  had  a  more  positive  effect  on  changes  of  in- 
structional behavior  of  instructors  of  short-term,  non-credit  courses  as 
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measured  by  subsequent  student  ratings.   This  conclusion  is  evidenced  by 
a  statistical  analysis  result  indicating  the  reverse  direction  as  pre- 
dicted by  Hypothesis  5  and  the  rejection  of  Hypothesis  6.   Although  in- 
structors teaching  long-term  courses  had  a  longer  amount  of  time  to 
change  instructional  behavior,  instructors  teaching  short-term  courses 
seemed  to  have  valued  student  ratings  of  instruction  more.   This  could 
possibly  explain  the  result  that  part-time  instructors  teaching  short- 
term  courses  received  higher  end-of-term  student  ratings  means  on  most 
item  variables  than  did  part-time  instructors  teaching  long-term 
courses . 

Hypotheses  2,  3,  and  4  investigated  the  second  dependent  variable 
of  instructor  self -ratings .   From  the  linear  regression  and  t-statistic 
analyses  results  of  Hypothesis  2,  it  generally  would  appear  that  in  the 
feedback  condition,  instructor  self -overrating  was  not  a  valid  predictor 
of  change  in  end-of-term  student  ratings  for  part-time  instructors. 
Other  results  of  instructor  self-ratings  testing  Hypotheses  3  and  4  in- 
dicated that  instructors  who  self -overrated  (rated  themselves  higher 
than  their  students)  at  mid- term  tended  to  rate  themselves  lower  at  the 
end-of-term;  and  instructors  who  self-underrated  (rated  themselves  lower 
than  their  students  rated  them  at  mid- term)  tended  to  rate  themselves 
higher  at  the  end-of-term.   Instructors  in  the  two  mid- term  feedback 
conditions  received  a  written  comparison  summary  of  student  ratings  with 
self -ratings,  while  the  instructors  in  the  one  no- feedback  group  who 
self-rated  at  mid-term  did  not.   This  result  seems  to  suggest  that  the 
practice  of  instructor  self-rating  may  have  had  a  significant  effect  on 
increasing  the  instructor  awareness  of  his/her  own  teaching  practices. 
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Hypothesis  6  investigated  differences  between  opinions  instructors 
of  long-term  courses  and  instructors  of  short-term  courses  had  towards 
student  ratings.   The  analysis  of  the  t- statistic  data  seems  to  suggest 
that  instructors  of  short-term,  non-credit  courses  tended  to  value  stu- 
dent ratings  data  more  highly  than  instructors  of  long-term,  credit 
courses,  resulting  in  a  rejection  of  Hypothesis  6. 

Of  the  three  instructor  demographic  characteristics  tested  in  this 
study,  results  indicated  that  instructor  teaching  experience  is  a  criti- 
cal variable  in  relation  to  the  student  ratings  of  part-time  faculty. 
The  statistical  analyses  clearly  showed  that  the  less  experience  the 
part-time  instructor  reported,  the  lower  his/her  student  ratings  of  in- 
struction were  likely  to  be.   Instructor  sex  and  educational  degree  ob- 
tained did  not  appear  as  related  variables  to  student  ratings. 

There  also  appeared  to  be  a  large  significant  difference  between 
the  motivation  levels  of  students  enrolled  in  short-term  courses  (non- 
credit  courses)  and  students  enrolled  in  long-term  courses  (credit 
courses).   Students  who  enrolled  in  the  short-term,  non-credit  courses 
tended  to  be  more  highly  motivated. 

Implications 

The  primary  purpose  of  the  study  was  to  determine  the  effect  dif- 
ferent forms  of  student  ratings  feedback  had  on  subsequent  part-time 
faculty  student  ratings  and  instructor  self -ratings .   Part-time  faculty 
opinions  towards  student  ratings  of  instruction  also  were  explored. 

Although  found  non-significant  at  the  .05  level,  possibly  the  most 
important  implication  of  this  study  was  the  apparently  modest  improve- 
ment student  ratings  feedback  appeared  to  have  had  on  the  subsequent 
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student  ratings  of  part-time  faculty.   The  results  of  this  study  were 
consistent  with  earlier  studies  by  Aleamoni  (1978),  Centra  (1973a, 
1977),  McKeachie  et  al.  (1971),  and  Sullivan  and  Skanes  (1974)  with  one 
exception.   Augmented  mid- term  student  ratings  feedback  (printed  feed- 
back with  oral  consultation)  appeared  not  to  have  been  more  sig- 
nificantly effective  in  changing  certain  instructional  behavior  of  part- 
time  faculty  than  was  printed  mid- term  ratings  feedback.   These  findings 
did  not  confirm  an  earlier  study  by  Aleamoni  (1978)  which  showed  the 
positive  effect  of  augmented  student  ratings  feedback  over  printed 
ratings  feedback  on  instructional  changes.   One  major  difference  is  that 
Aleamoni's  study  consisted  of  full-time  faculty  teaching  credit  courses. 
Student  evaluation  of  instruction  is  a  widely  accepted  and  utilized 
evaluation  instrument  of  credit  courses.   Although  most  research  studies 
of  the  validity  and  reliability  of  student  ratings  have  focused  on  its 
use  with  full-time  faculty  teaching  credit  courses,  it  has  been  widely 
adopted  as  an  instructional  evaluation  tool  with  part-time  faculty 
teaching  both  credit  and  non-credit  courses.   This  evaluation  instru- 
ment, adopted  from  traditional  educational  settings  and  implemented  to 
evaluate  the  instruction  of  an  emerging  new  type  of  instructor  to 
American  higher  education  (part-time  faculty)  teaching  a  new  learning 
structure  (non-credit  courses),  deserves  further  research  investigation. 

Further  implications  from  this  study  suggest  that  not  only  does 
student  ratings  feedback  help  improve  subsequent  part-time  faculty  stu- 
dent ratings,  but  it  appears  also  to  have  had  a  more  positive  effect  on 
instructors  of  short-term,  non-credit  courses.   This  finding  was  in  the 
reverse  direction  than  predicted.   The  hypothesis  prediction  was  based 


89 

on  a  study  by  Centra  (1973b)  in  which  he  indicated  that  student  ratings 
feedback  did  effect  some  changes  in  student  ratings  over  time.   There- 
fore, the  investigator  predicted  that  student  ratings  feedback  would 
have  a  more  positive  effect  on  the  student  ratings  of  instructors  of 
long-term  courses  than  the  instructors  of  short-term  courses.   The  basis 
for  the  reasoning  was  that  behavioral  change  requires  time,  and  a  longer 
time  lapse  existed  in  this  study  between  the  student  ratings  feedback 
point  and  the  end-of-term  evaluation  point  for  instructors  of  long-term 
courses  (six  weeks)  than  short-term  courses  (two  to  three  weeks). 
Results  from  this  present  study  indicated  the  reverse  situation, 
however.   This  finding  may  be  the  result  of  a  "halo"  effect  since  the 
instructors  of  short-term  courses  in  the  study  had  never  before  received 
student  ratings  feedback  from  the  employing  institution  and  were  aware 
of  the  research  nature  of  the  study.   Perhaps  the  sheer  presence  of  a 
mid- term  evaluation  procedure  in  a  non-credit  course  was  sufficient  to 
indirectly  suggest  to  the  class  and  the  instructor  the  value  of  instruc- 
tional quality  to  the  educational  institution.   Another  possible  ex- 
planation of  the  results  is  that  continual  employment  of  part-time 
faculty  teaching  short-term  courses  is  more  tenuous  in  nature  and  offers 
less  job  security  than  faculty  teaching  long-term  courses.   This  is  be- 
cause students  typically  enroll  in  non-credit  courses  on  an  elective 
basis  as  opposed  to  students  in  long-term  courses  who  often  enroll  in 
these  courses  as  requirements  in  their  curriculum  of  study.   Therefore, 
the  non-credit  nature  of  the  short-term  courses  in  the  study  is  heavily 
dependent  upon  students  who  are  satisfied  with  the  instruction  received. 
Consequently,  in  order  to  hold  on  to  their  jobs  on  a  continuous  basis. 
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enrollment  in  non-credit  courses  must  be  maintained.   Sustanence  of 
course  enrollment  levels  are  attained,  in  part,  by  part-time  faculty  who 
are  aware  and  concerned  about  the  degree  of  student  satisfaction  of  in- 
struction.  Therefore,  part-time  faculty  teaching  short-term,  non-credit 
courses  possibly  pay  closer  attention  to  student  satisfaction  levels 
than  do  the  part-time  faculty  of  longer  term  courses.   This  also  may  ex- 
plain the  findings  which  seemed  to  indicate  that  instructors  of  short- 
term,  non-credit  courses  valued  student  opinion  of  instruction  more  than 
did  instructors  of  long-term,  credit  courses. 

Another  possible  explanation  for  the  greater  effect  of  student 
ratings  on  short-term  course  instructors  may  be  due  to  the  motivation 
levels  of  the  students.   The  results  of  the  post  hoc  analyses  of  student 
motivation  levels  confirmed  that  students  in  short-term,  non-credit 
courses  were  more  highly  motivated  to  enroll  in  a  particular  course  than 
students  of  long-term  courses.   This  result  was  also  consistent  with 
some  informal  observations  of  the  researcher  who  noted  the  more  intense 
questioning  of  the  students  enrolled  in  short-term  courses  about  the 
reason  for  and  appropriateness  of  the  evaluation  procedures.   It  seemed 
that  the  long-term,  credit  course  students  were  more  accustomed  to  stu- 
dent evaluation  of  instruction  procedures  than  were  the  students  of  the 
short-term  courses.   The  long-term  course  instructors  also  were  accus- 
tomed to  student  opinion  ratings,  since  this  is  a  dominant  and  well- 
accepted  evaluation  tool  to  them.   Although  both  short-term  and  long- 
term  course  students  were  encouraged  by  the  investigator  to  answer  all 
items  on  the  evaluation  form  to  the  best  of  their  ability,  anecdotal 
remarks  by  some  short-term  course  students  indicated  more  difficulty  in 
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answering  some  items  that  "just  didn't  fit  the  course."   Consequently, 
although  the  evaluation  instrument  had  a  forced- choice  format,  there  may 
have  been  more  response  item  omissions  from  the  short-term  course  stu- 
dents, and  this,  in  turn,  may  have  affected  the  ratings  data  of  the 
short-term  courses.   Another  possible  explanation  of  the  apparently 
stronger  effect  of  mid-term  student  ratings  feedback  on  end-of-term 
ratings  of  part-time  instructors  of  short-term  courses  is  that  the  in- 
structors of  short-term  courses  teach  a  more  highly  motivated  audience 
who  question  and  challenge  their  instruction  more  than  students  enrolled 
in  long-term,  credit  courses.   This,  in  turn,  would  tend  to  give  the  in- 
structor more  informal  student  feedback  specificity  of  his/her  teaching 
behavior  on  a  continuous  basis,  thus  facilitating  and  expediting  in- 
structional change. 

Results  of  the  linear  regression  and  t-statistic  analysis  suggest 
that  the  dependent  variable  of  instructor  self -overrating  is  not  a  valid 
predictor  of  end-of-term  student  ratings.   This  result  was  not  consis- 
tent with  Centra's  1973b  study  in  which  he  found  that  according  to  equi- 
librium theory,  student  ratings  feedback  produced  changes  in  instructors 
who  rated  themselves  more  favorably  than  their  students  rated  them. 
Perhaps  in  this  study,  the  difference  in  instructor  self- ratings  and 
student  ratings  in  the  feedback  condition  was  not  large  enough  to 
produce  a  discrepancy  impact  of  necessary  magnitude  to  motivate  instruc- 
tors to  change  their  teaching  practices. 

Further  results  of  the  analyses  of  the  dependent  variable,  instruc- 
tor self -ratings,  do  imply  a  confirmation  of  the  equilibrium  theory, 
however.   The  result  of  the  change  in  instructor  end-of-term  self-rating 
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for  both  instructor  self -underraters  and  instructor  self -overraters 
seems  to  indicate  the  effects  of  discrepancy  on  self -ratings .   Based  on 
equilibrium  theory,  the  instructor  faced  with  a  discrepancy  of  either 
overrating  or  underrating  his/her  teaching  performance  was  motivated  to 
achieve  "balance"  or  "consonance"  with  reality.   This  seemed  to  imply 
that  the  instructors  who  underrated  themselves  in  comparison  to  student 
ratings  were  motivated  to  rate  themselves  higher  on  the  end-of-term 
evaluation,  and  instructors  who  overrated  themselves  in  comparison  to 
student  ratings  were  motivated  to  rate  themselves  lower  on  the  end-of- 
term  evaluation.   Self-ratings  implications  for  achieving  increased  in- 
structor awareness  of  his/her  own  teaching  practices  are  obvious  and 
could  prove  to  be  a  useful  and  effective  tool  for  part-time  faculty 
evaluation  when  utilized  in  conjunction  with  student  ratings  of  instruc- 
tion and  instructional  development  programs. 

The  t- statistic  analyses  of  part-time  instructor  responses  to  the 
Instructor  Questionnaire  was  significant  at  the  .05  level  on  two  of  the 
three  items  which  measured  instructor  opinions  of  student  ratings  of  in- 
struction.  Findings  showed  that  instructors  of  short-term  courses 
placed  higher  value  on  student  ratings  of  instruction  than  did  instruc- 
tors of  long-term  courses.   Implications  are  that  part-time  instructors 
teaching  short-term,  non-credit  courses  seemed  to  value  more  knowing  how 
satisfied  their  students  are  and  how  their  students  perceive  their 
teaching  behavior;  therefore,  whenever  possible,  student  ratings  of  in- 
struction should  be  used  as  an  evaluation  tool.   Since  student  ratings 
of  instruction  are  a  credible  and  reliable  source  of  information,  they 
could  be  utilized  effectively  in  conjunction  with  instructional  develop- 
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ment  programs  to  improve  the  teaching  behavior  of  part-time  faculty 
teaching  non-credit  courses. 

The  results  of  the  instructor  demographic  variables  imply  that  sex 
and  educational  degree  obtained  do  not  appear  to  be  related  to  the  level 
of  student  ratings.   This  is  consistent  with  the  results  of  earlier 
studies.   However,  student  ratings  data  suggest  that  instructor  teaching 
experience  has  a  significantly  strong  relationship  to  the  teaching  per- 
formance of  part-time  faculty,  and  this  is  consistent  with  previous  re- 
search reported  by  Centra  (1977).   Results  seem  to  imply  that  the  least 
effective  part-time  instructors  are  those  who  have  the  least  amount  of 
teaching  experience.   Centra  (1977)  found  that  some  of  the  lowest  stu- 
dent ratings  tended  to  occur  during  the  instructor's  first  three  years 
of  teaching.   Implications  for  employment  decisions  by  educational  in- 
stitutions are  obvious;  wherever  possible,  institutions  should  try  to 
employ  part-time  faculty  with  some  prior  college  or  adult  education 
teaching  experience.   According  to  this  study,  the  number  of  years 
teaching  experience  is  a  critical  variable  among  other  criteria  to  be 
considered  when  deciding  to  hire  a  part-time  instructor  for  a  particular 
area  of  expertise. 

Recommendations  for  Further  Research 

As  a  result  of  this  study  several  recommendations  for  further  re- 
search are  in  order.   These  recommendations  involve  five  main  areas,  the 
appropriateness  of  student  ratings  feedback  of  part-time  faculty  as  an 
evaluation  tool,  augmented  student  ratings  feedback,  instructor  self- 
ratings,  characteristics  and  motivation  of  students  enrolled  in  non- 
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credit  courses,  and  the  attitude  of  part-time  instructors  toward  student 
ratings . 

One  recommendation  of  this  study  is  that  further  attempts  be  made 
through  research  to  discern  more  clearly  the  appropriateness  of  student 
ratings  of  instruction  as  a  valid  evaluation  tool  for  part-time  faculty, 
especially  those  instructors  in  a  non-credit  learning  situation.   Al- 
though student  ratings  feedback  appeared  to  have  a  more  positive  effect 
on  the  student  ratings  of  part  time  faculty  teaching  non-credit  courses, 
some  of  the  items  contained  in  the  evaluation  tool  used  in  this  study 
seemed  to  the  students  to  be  inappropriate  and  at  times  were  cause  for 
concern.   Even  with  thorough  explanation  from  the  investigator  about  the 
nature  and  intended  use  of  the  evaluation  form,  the  students  enrolled  in 
short-term,  non-credit  courses  appeared  to  be  more  apprehensive  in 
answering  certain  items  than  were  the  students  enrolled  in  long-term, 
credit  courses.   This  appeared  to  be  the  case  even  though  the  evaluation 
instrument  had  a  forced-choice  format. 

A  second  consideration  concerns  the  need  for  continued  research  to 
further  investigate  the  effect  that  different  forms  of  augmented  feed- 
back might  have  on  part-time  instructional  behavior.   Perhaps  a  study 
with  more  than  two  kinds  of  student  ratings  feedback  procedures  of  vary- 
ing length  and  format  (group  versus  individual,  for  example)  would  more 
clearly  distinguish  effective  feedback  modes,  and  the  amount  and  inten- 
sity of  feedback  desirable  to  effect  part-time  faculty  student  ratings 
and,  subsequently,  teaching  behavior. 

A  third  recommendation  is  for  further  research  in  the  use  of  in- 
structor self-ratings  data  as  a  catalyst  for  increased  instructor  aware- 
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ness  of  teaching  behavior.   The  results  of  this  study  seemed  to  imply 
this  for  instructors  of  both  the  feedback  and  no -feedback  conditions. 
An  interesting  investigation  would  be  to  determine  if  the  magnitude  of 
instructor  awareness  of  teaching  performance  varies  with  either  formal 
feedback  conditions,  informal  feedback  conditions,  or  with  different 
types  of  learning  structures  such  as  non-credit  courses,  for  example. 

A  fourth  recommendation  involves  further  investigation  into  the 
differences  in  student  characteristics  and  motivation  of  non-credit  stu- 
dents as  compared  to  credit  students  who  enroll  in  classes  taught  by 
part-time  faculty.   If  such  student  characteristic  differences  exist,  do 
these  differences  contribute  to  part-time  instructors  of  non-credit 
courses  receiving  more  informal  feedback  from  students  in  the  classroom 
regarding  their  own  teaching  behavior  than  part-time  instructors  of 
credit  courses? 

A  final  recommendation  for  further  research  concerns  the  evaluation 
of  part-time  faculty  opinion  towards  student  ratings  as  an  evaluation 
tool.   This  study's  results  led  to  the  conclusion  that  part-time  faculty 
teaching  short-term,  non-credit  courses  valued  student  ratings  of  in- 
struction more  than  part-time  faculty  teaching  long-term,  credit 
courses.   Perhaps  further  investigation  utilizing  a  questionnaire  with  a 
seven-point  Likert-type  scale  could  further  discern  and  distinguish  any 
differences  of  part-time  instructor  opinions  towards  student  ratings. 
Is  the  source  of  these  differences,  for  example,  an  instructor  charac- 
teristic, due  to  the  type  of  course  taught  (credit  versus  non-credit), 
or  is  it  perhaps  institutionally  related? 
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APPENDIX  A 


INSTRUCTIONAL  DEVELOPMENT  EFFECTIVENESS  ASSESSMENT  (IDEA') 
STANDARD  FORM  (MID -TERM) 


inrA  Survey  fonti  --  Student  «eact1ons   to   Instruction  and  Course; 
Your  thoughtful   ans.ers   to  these  questions  will   provide  helpful    infomatlon   to  your   mstr 
I    .Describe   the  frequency  of  your   instructor's    teaching  procedures,   using   the 


fol lowing  code: 

1 -HARDLY  EVER  -Z-OCCASSIONALLY     3-SOMETIHES     4-FREQUENTLY     5-AlKDST  Al'wAYS       | 


The  Instructor: 

1  ProTOtes   teacher-student  discussion   (as  opposed   to  «re  responses    to  questions). 

?'  Finds  ways  to  help  students  answer  their  own  questions.     ■ 

3'  Encourages  $tud«nts  to  express   themselves  freely  and  openly. 

4  Seem  enthusiastic  about  the  subject  matter. 

5  Changes  approaches  to  neet  new  situations.  ... 
s'  Gives  exaninations  which  stress  unnecessary  memorization. 
7"  Speaks  w1t)i  exprMsiveness  and  variety  in  tone  of  voice. 

8  r-^nltrates   t^e   l-port^nce  and  significance  of  the  subject  matter. 

9  Kanes  presentations  which  are  dry  and  dull. 

]]■  ^;:::r:^'r<r.rjir>;  t:.TJ,z  "Ct-  =-; » ..  .»<or,,=,  <, 

It'  «,«uri7»«  iait«r«al    in  a  Banner  which  aides   retention. 

is:  l^lltel  ""enti  W  tnUllectual   effort  beyond  that  required  by  «st  courses, 

16  Clearly  sutcil  the  objectives  of  the  course. 

v.  Ex"ain,  cou«e  material   clearly,   and  explanations  are  to  the  point. 


18. 
19. 
20. 


Relates  course  material   to  real    life  situations.  , 

Gives  examination  questions  which  are  unreasonably  deUiled  (picky). 
Introduces  stl^ilatinfl  ideas  about  the  subject. 


.  On  each  of  the  objectives   listed  belo-.  rate  ,^^  P™'!"*  ^"  *'■'  ~i;,"?tror'';'-.th«''' 
coapared  with  that  wde  in  other  courses  you  have  Uken  at  this  university  or  a   .^r,. 
university.       In  this  course  ny  progress  so  far  is: 

1-LOW  (lowest  10  percent  of  courses  I  have  taken  here) 

2-LOW  AVERAGE   (next  20  percent  of  courses) 

3. AVERAGE   (middle  40  percent  of  courses) 

4-HIGH  AVERAGE  (next  20  percent  of  courses) 

5-HIGH  (highest  10  perc»(nt  of  courses)  


'rogress  on: 

21.     Gaining  factual   knowledge  (ter,»inologjr.  classifications,  -etbods.   trends) . 

\l:     trarn!:5  [^I^J "iu^sr^^i-a^'-'iiil^trrrtl'o-nl/Snirng.   p.ble.-solv1ng  and 

SllSnrSSif^c  Skills,  co-petencles  and  points  of  view  needed  by  professionals   In 

S:r:i'ni^r;r4l°e"IJna^l*i'n^h^s';i'.l";o"!bout  the  process  of  gaining  new  knowledge. 

"•     ^:i\ll\:i  r"n;:VfTr«ia"-responsib111ty  (self-reliance,  self  f*-1pl'-j- 
II:     SlnlnS  a  broaSer  unde?sunding  and  appreciation  of  Intellectual -cul  tural   activity 

(nwsic.   science,   literature,  etc.). 
29      Oevelopinq  skill   in  expressing  myself  orally  or  in  writing. 
30:     Oilcover?ng  the  i«vlicat1ons  of  the  course  material   for  undersUndlng  m^i^f 

(interests,  talents,  values,  etc.). 


24. 
25. 


»  On  the  next  four  questions,  compare  this  course  with  others  you  have   taken  at  this 
institution,  using  the  foUowing  code: 

1-WJCH  LESS  THAN  MOST  COURSES     2-LESS  THAN  WST     3-AfiOUT  AVERAGE 

4.M0RE  THAN  «ST     5-MUCH  MORE  THAN  MOST  


The  course: 

31:     Aaount  of  reading.  ,      »         , 

32      AMunt  of  work  in  other  (non-reading)  ass1gn«nis. 

U:     S:;:;"i;^hIcrtirco:r::rangs  together  (various  topics  and  class  activities   are 
related  to  each  other). 
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Describe  your  attitude  tnd  behavior  In  this  course,  using  the  following  code: 
1-DEFINITELY    2-MORE  FALSE  THAN  TRUE   3-IN  BETWEEN    4-MORE  TRUE  THAN  FALSE 
5-DEFIHITELY  TRUE 


SeU-ratIng  so  far: 

35.  I  have  worked  harder  1n  this  course  than  on  oust  courses  I  have  taken. 

36.  I  had  a  strong  desire  to  take  this  course. 

37.  I  would  like  to  take  another  course  from  this  Instructor. 

38.  As  a  result  of  taking  this  course.  I  daw  nore  positive  feelings  toward  this  field 
of  study. 

39.  Leave  this  space  blank.. 

A.  Blacken  space  nunber  1  on  the  Response  Card. 


•For  the  following  six  questions,  B-6,  describe  the  frequency  of  your  Instructor's 
teaching  procedures,  using  the  following  code: 

l-HARDLY  EVER    2-OCCASIONALLY    3-SOMETIMES    4.FRE0UENTLY    5-ALHOST  ALWAYS 


The  Instructor: 

B.  Relates  various  topics  and  class  activities  to  each  other. 

C.  Discusses  the  cultural  and  tfthetic  values  of  the  subject. 

0.  Relates  the  course  materials  to  other  courses  and  academic  fields. 

E.  Is  willing  to  spend  extra  tin  with  students. 

F.  Cares  about  students  as  people. 
6.  Shows  Interest  In  student  Ideas. 


•Please  answer  these  additional  questions  using  the  following  responses: 
1-STRONGLY  DISAGREE    Z-DISAGREE    3-NEITHER  AGREE  OR  DISAGREE    4-AriREE 
5-STRONGLY  AfiREE 


40.  Compared  to  others  I  have  had,  this  Instructor  Is  one  of  the  most  effective. 

41.  Overall,  this  course  Is  of  considerable  value  to  me. 


APPENDIX  B 


INSTRUCTIONAL  DEVELOPMENT  EFFECTIVENESS  ASSESSMENT  (IDEA) 
STANDARD  FORM  (END -OF -TERM  EVALUATION) 


SURVEY  fORM  --  SOiDBJ  R64CTK>4S  TO  INSTRUCTION  /ND  COURSES 


Your  tnoughllul  aniw»rs  10  tlWM  qu««llon»  olll  pfO.W*  h»)ptul 
Intormation  to  younnslnjctof. 


•  D«senb«  ih«  frtqutncy  ol  your  intlnietof »  iMChinfl  p»oc»- 
duras.  using  Iht  following  cod*: 

I— Hardly  E»tr  3— Som«tlm«« 

2— Occasionally       4— Fraquanily        5— Almoal  Alwiya 


Tha  Instructor 

1  oromoiea    ieacnorstuaent   alscuasion   las   opoosea   to   mare 
responses  to  Questions). 

2  Pouno  wavs  to  neiD  students  answer  tf^eir  own  questions. 

3.  Encouraqeo  students  10  express  tnemseives  freely  and  openly. 

4  Seemed  enthusiastic  apout  mo  supject  matter. 

5.  Changed  approaches  to  meet  new  situations. 

*  Gave  examinations  which  stressed  unnecessary  memorization. 

7  SpoKe  with  expressiveness  and  variety  m  tone  ot  voice. 

a  Demonstrated  the  imponance  ana  significance  of  the  subiect 

mailer 
9.  Made  presentations  which  were  dry  and  dull. 

10  Made  It  Clear  now  each  topic  fit  into  the  course. 

11  Explained  me  reasons  lor  criticisms  ot  students'  academic  per- 
iprmance. 

12  siave  examination  questions  wntch  were  unclear 

13.  Encouraqea  student  comments  even  when  mey  turned  out  lo  P* 

incorrect  or  irrelevant 
1  i  Summarized  material  m  a  manner  which  bided  retention. 
15  Shmuiatea  students  to  intellectual  effort  Peyond  that  required  by 

most  courses. 
1 6.  Clearly  slated  me  ooiecnves  of  the  course. 
17  Explained  course  material  clearly,  and  explanations  were  to  the 

DOinl. 
18.  Reiaieo  course  material  to  real  life  situations. 

19  Gave  examination  questions  which  were  unreasonably  detailofl 
ipicKyi. 

20  Introduced  stimulating  ideas  aPout  me  suPject. 


•  On  eacn  ol  the  oblectlvea  Haled  below,  rale  the  pragreaa  you 
have  made  in  tnia  coorae  compared  with  that  made  in  ollwr 
courses  you  have  taaen  ai  Ihia  eo(l»g*  of  untoenity.  In  thla 
course  my  progroaa  was: 

i_l.aw  lloweai  10  percent  ol  couree*  I  Han  taken  IM>«) 
2— LOW  Average  (next  20  percent  ol  cour***) 
3— Average  imlddle  40  percent  ol  course*) 
4— HIgn  Average  (nesi  20  p*rc*nt  ol  ceursas) 
S— Hlgli  (hlgn*sl  10  p*rc*nl  ol  cours**) 


Progress  on: 

21  Gaming  factual  knowledge  (terminology,  claasiflcatlona.  rrwth- 
ods,  trends). 

22.  Learning  lunoameniai  principles,  generalizations,  or  theories. 

23.  Learning  to  apply  course  material  tp  improve  rational  thinking, 
orooiem-soiving  and  decision-mahing. 

24  Developing  specific  skills,  competencies  and  points  ol  view 
needed  Dy  prolesaionaia  in  the  field  most  closely  related  to  this 
course. 

25.  Learning  now  prolessionais  m  this  field  go  about  the  process  ot 
gaming  new  knowledge. 

26.  Developing  creative  capacities. 

27  Developing  a  sense  of  personal  responsibility  (selt-re)l»nc*,  self- 
discipline) 

28.  Gaming  a  Broader  understanding  and  appreciation  ol  intellectual- 
cultural  activity  imusic,  science,  literature,  etc.). 

29  Developing  sxiil  m  expiessing  mysell  orally  or  in  writing. 

30.  Discovering  me  implications  ol  the  course  material  for  under- 
standing myself  (interests,  talents,  values,  etc.). 


•  On  Ihe  next  lour  queallona.  compare  this  course  with  others 
you  nave  taken  at  this  Institution,  using  the  following  code: 

1  —Much  Less  than  Most  Courses 

2— LeaathanMost 

3— About  Average 

4— More  than  Most 

S— Much  More  than  Moal 


The  Course: 

31  Amount  ol  reading. 

32.  Amount  ot  work  in  other  (non-reading)  assignments. 

33.  Diftlcuity  of  subiect  matter 

34.  Degree  lo  whicn  the  course  nung  together  (various  topics  and 


class  activities  V 


}  related  to  eacn  othen. 


•  OMcrttM  your  altitudes  toward  and  behavior  In  thia  course. 

using  the  lollowing  code: 
1—0*llnllely  False 

2— Mors  Fala*  than  True       4— More  True  then  False 
3— In  Between  5— Oellnilely  True 


Sell-rsling: 

35. 1  worked  harter  on  inis  course  than  on  most  courses  I  have  taken. 

36. 1  had  a  strong  desire  to  lake  this  course. 

37. 1  would  like  to  take  another  course  from  this  instructor 

38.  As  a  result  ol  taking  this  course,  I  have  more  positive  feelings 
toward  this  field  of  study 

39.  Leave  this  space  oiank.  Continue  with  question  A. 

A.  Blacken  space  numper  i  on  the  Response  Card. 


•  For  tlM  lollowing  sl>  queatlone.  S-<3.  describe  the  Irsquency 
ot  your  Instructor's  teaching  procedures,  using  the  lollowing 
cod*: 

1— Hardly  Ev*f         3— Sometimes 

2— Occasionally       4— Frequently        5— Almoet  Alweys 


Th*  Instructor 

a.  Rotated  various  topics  and  class  activities  to  eacn  other. 

C.  Dlscuss*d  the  cultural  and  esthetic  values  of  the  subject. 

0.  Related  the  cpurse  matenals  to  other  courses  and  academic 

fieus. 
E.  Was  willing  to  sp«nd  extra  time  with  students. 
F  Cared  about  students  as  p*ool*. 
G.  Showed  interest  in  student  id**a. 


?iease  answer 


aaU'onal   questions  using  tti* 


ini]   resDonseS 


1  -    Strongly   Disagree 

2  -    Disagree 

3  -  .Seitner  «gree  or  Disagree 

4  -    Agree 

5  -    Strongly  Agree 

Compared  to  others   I  fiave  fiad,   this   instructor 
beenone  of  the  most  effective. 

Overall,   this  course  was  of  considerable  value 
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APPENDIX  C 


INSTRUCTIONAL  DEVELOPMENT  EFFECTIVENESS  ASSESSMENT 

SURVEY  ITEM  RELIABILITIES,  STANDARD  DEVIATIONS,  AND 

STANDARD  ERRORS  OF  MEASUREMENT 


Item 


Item  Reliabilities,  Standard  Deviations  and  Standard  Errors  of 
Measurement  for  IDEA  Standard  Form 


Snail  Classes  (10  raters) 
N  =  440 

SD   SE. 


sb 


ment 


65 

79 

67 

31 

2 

.50 

.67 

.53 

.36 

3 

.53 

.69 

.51 

.34 

4 

.53 

.73 

.55 

.29 

.50 

.57 

.61 

.3b 

6 

.40 

.57 

.74 

.49 

7 

.51 

.75 

.66 

.32 

8 

.46 

.63 

.51 

.31 

9 

.56 

.72 

.69 

.3; 

10 

.43 

.60 

.58 

.3/ 

11 

.44 

.51 

.73 

.49 

12 

.52 

.68 

.74 

.42 

13 

.44 

.51 

.69 

.43 

14 

.45 

.62 

.58 

.42 

15 

.52 

.68 

.72 

.41 

16 

.45 

.52 

.50 

.3/ 

17 

.55 

.71 

.62 

:n 

18 

.60 

.75 

.63 

.34 

19 

.42 

.59 

.77 

.49 

20 

.62 

.77 

.53 

.33 

21 

.47 

.64 

.57 

.34 

22 

.36 

.53 

.55 

.38 

23 

.37 

.54 

.62 

.42 

24 

.50 

.67 

.66 

.38 

25 

.52 

.68 

./I 

.40 

26 

.60 

.75 

.71 

.36 

27 

.47 

.54 

.65 

.39 

28 

.58 

.73 

.87 

.4b 

29 

.69 

.32 

.84 

.36 

30 

.54 

.70 

.72 

.i<i 

31 

.73 

.88 

.81 

.28 

32 

.67 

.80 

.67 

.30 

33 

.56 

.30 

.61 

.2; 

34 

.41 

.53 

.55 

.36 

35 

.45 

.62 

.62 

.38 

36 

.54 

.70 

.74 

.41 

37 

.62 

.77 

.30 

.38 

38 

.56 

.72 

.66 

.3b 

39 

.22 

.36 

.38 

.30 

AVERAGE 

.59 

.3; 

Medium  Classes  (20  raters ) 
N  =  415 

SD   SE. 


sb 


ment 


.82  .90 
.74  .35 
.80  .89 
.85 
.32 
.54 
.37 


.74 
.59 
.47 
.77 
.72 
.74 
.55 


.59  .22 

.57  .22 

.60  .20 

.53  .21 


.59 
.67 
.53 
.84  .50 
.85  .68 
.79  .50 


.25 
.40 
.25 
.20 
.26 
.23 


.56  .72  .50  .32 
.50  .67  .66  .33 
.55  .79  .57  .26 


.55 


79  .54  .25 
56  .80  .60  .27 
69  .82  .56  .24 


.76 
.77 
.64 
.76 


.86  .62 
.37  .65 


.68 
.69 


.23 
.23 
.32 
.26 


.68  .81  .51 
.63  .77  .49 
.66 


.70 

.71 

.71 

.62 

.73 

.79  .38 

.71  .83 


.80  .53 
82  .51 


22 
23 

24 
22 

60  .25 

61  .25 


83 

83 

77  .55  .26 

.72  .29 

.75  .25 

.61  .25 


.84 


78 


.86  .92 

.78  .88 
.78 
.54 

.66  .80 

.74  .85 

.78  .88 

.69  .82 

.28  .44 
.31 


.76 
.64 
.56 
.48 


.21 
.22 
.19 
.23 


.58  .26 


.27 
.25 
.26 
.22 

.25 


Large  Classes  (40  raters, 
N  =  226 

SD   SE. 


sb 


'ment 


.91  .95  .32 

.37  .93  .62 

.39  .94  .59 

.85  .92 

.31  .90 


.64  .73 

.87  .93 

.33  .91 

.33  .91 


.13 
.15 
.17 
.20 
.17 
.27 
.13 
.15 
.18 


.32  .90  .43  .15 


.70 
.79 
.79 
.76 
.73 
.75 
.65 
.34 
.64 


81 

82  .58 

89  .58 


89  .57 

90  .56 


.51  .22 
25 
19 
19 


91 
92 
90  .62 
86  .62 
92  .60 


18 

55  .17 

59  .17 


82 


.45 
.41 
.51 
.50 
.50 
.54 
.46 
.56 
.51 
.55 


.15 
.17 
.13 
.17 
'.19 
.19 
.17 
.21 
.13 
.26 


.92  .96 
.91  .95 


.39 

.73  .84 

.85  .92 


59 

56 

94  .60 


14 
15 
15 

43  .17 
53  .16 


.84 
.86 


.54 


.91  .61 

.92  .75 

.89  .58 

.70  .23 
.39 


.18 
.21 
.19 
.13 
.13 


•^OTES:  r  -  split-half  correlations,  e.g..  based  upon  b  ot  lU  raters,  r^^  -  sput-nalf 
°  correlations  corrected  by  Spearman-Brown  formula.   ,  ^  ^    ,  »  „,,„ 

Source-  THFA  Technical  Report  No.  J:   "Description  of  IDEA  Standard  Form  Data  Ba  e 

Manhatten.  KS :  Center  for  Faculty  Evaluation  and  uevelopment.  Kansas  State 
University,  1978. 
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APPENDIX  D 
INSTRUCTOR  QUESTIONNAIRE 


Division  of  Lifelong  Learning 

Instructor  Questionnaire 

Your  answers  to  these  questions  will  provide  helpful  information  to 
the  Division  of  Lifelong  Learning. 

On  each  question  below,  please  circle  your  answers  using  the  fol- 
lowing code: 

12  3  4  5 

very  little    little     average     somewhat   very  much 

1.  How  much  importance  do  you  think  the  University  of  South  Florida 
places  on  student  evaluations? 


2.  How  much  importance  do  you  think  the  University  of  South  Florida 
should  place  on  student  evaluations  of  instruction? 


3.  How  much  weight  do  you  feel  student  evaluations  of  instruction 
should  carry  with  your  overall  evaluation  as  an  instructor  in  the 
Division  of  Lifelong  Learning? 


4.  How  useful  do  you  feel  the  results  of  student  ratings  of  instruction 
are? 


5.  How  much  importance  do  you  place  on  knowing  how  satisfied  your 
students  are  with  your  teaching? 


Instructor  Name: 
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APPENDIX  E 
COVER  MEMORANDUMS 


University  of  South  Florida 
School  of  Extended  Studies 

Memorandum 


DATE:  February  1 ,  1986 

TO-  All  Instructors  of  the  Division  of  Lifelong  Learning 

FROM:  Lee  Leavengood,  Director 

SUBJECT:  Student  Evaluation  of  Instructor  Study 

The  School  of  Extended  Studies  is  participating  in  a  very 
important  school-wide  study  investigating  new  techniques  for- 
gathering student  ratings  of  instruction  data.  This  study  is 
extremely  vital  since  it  will  give  the  University  and  the  School 
of  Extended  Studies  new  insights  into  the  effect  of  student  ratinas 
on  teaching  behavior.  Of  particular  interest  in  this  investigation 
is  what  students  are  able  to  evaluate  in  the  classroom  and  how  useful 
this  information  might  be  to  the  instructor.  Your  full  participation 
is  needed  and  appreciated  in  this  study  since  its  results  are  critical 
to  the  future  planning  of  the  Division  of  Lifelong  Learning. 

Incorporated  into  this  study  will  the  the  "piloting"  of  a  new 
student  evaluation  form  which  will  be  administered  in  each  class 
by  a  course  coordinator.  Most  classes  will  have  this  form 
administered  twice  during  this  semester;  once  at  mid-term  and  once 
at  the  end  of  the  term,  and  some  only  at  the  end  of  the  term, 
depending  upon  which  group  you  are  assigned.  In  any  case,  you  have 
my  complete  assurance  that  the  strictest  confidence  will  be  kept, 
and  that  only  you  and  the  course  coordinator  will  see  your  individual 
student  ratings  report. 

Please  find  enclosed  a  short  instructor  questionnaire  regarding 
student  evaluation  of  instruction  for  you  to  fill  out.  You  are  also 
requested  to  choose  specific  course  objectives  for  your  class  and 
rate  their  importance  on  the  enclosed  Faculty  Information  Card  II 
(purple).  Please  return  the  completed  instructor  questionnaire 
and  the  Faculty  Information  Card  II  by  February  15.  1986,  in  the 
enclosed  envelope. 

On  behalf  of  the  Division  of  Lifelong  Learning  let  me  thank 
you  for  your  participation  in  this  valuable  research  study. 


113 


114 

UNIVERSITY    OF    SOUTH    FLORIDA 

y^^<^  TAMPA  •  ST.  PETERSBURG  •  FORT  MYERS  •  SARASOTA 


February  5,  1986 


Dear  Saint  Leo  College  Educational  Services  Faculty: 


You  recently  received  a  memo  from  David  Koval  ,  Dean  of  Educational  Services, 
regarding  a  dissertation  research  study  in  which  Saint  Leo  College  will 
participate  during  this  current  term. 

Of  particular  interest  in  this  investigation  is  what  students  are  able 
to  evaluate  in  the  classroom,  and  how  useful  this  information  might  be 
to  you,  the  instructor.  Pivotal  to  the  study  will  be  a  student  evalution 
form  which  will  be  administered  either  by  me  or  an  assistant  in  each  class. 
Most  classes  will  have  this  form  administered  twice  during  this  semester; 
once  at  mid-term  and  once  at  the  end  of  the  term,  depending  upon  which  group 
you  are  assigned.  You  will  receive  a  phone  call  from  me  to  inform  you  of 
your  particular  course  evaluation  schedule.  In  any  case,  you  have  complete 
assurance  from  Saint  Leo  College  that  the  strictest  confidence  will  be  kept, 
and  that  only  you  will  see  your  individual  student  ratings  report. 

Please  find  enclosed  a  short  instructor  questionnaire  regarding  student 
evaluation  of  instruction  for  you  to  complete.  You  are  also  requested  to 
choose  specific  course  objectives  for  your  class(es)  and  rate  their  importance 
to  you  on  the  enclosed  Faculty  Information  Card  II  (purple).  Please  return 
the  completed  instructor  questionnaire  and  the  Faculty  Information  Card  II 
by  February  19,  1986  in  the  enclosed  stamped  envelope. 

On  behalf  of  Saint  Leo  College  Educational  Services,  let  me  thank  you 
for  your  participation  in  this  research  study. 


Sincerely, 
Cher/l  M.  Burbano 


■^^ 


APPENDIX  F 
ORAL  CONSULTATION  FEEDBACK  INSTRUCTIONS 


Course  Coordinator  Consultation  Instructions 
In  order  to  assure  uniform  procedures  in  consultative  feedback  to 
instructors,  please  adhere  to  the  following  guidelines  in  your 
conferences : 

1.  Telephone  the  instructor  the  day  after  the  mid- term  assessment  and 
set  up  a  conference  appointment  within  one  week. 

2.  Make  the  instructor  feel  comfortable,  offer  coffee  and  ask  generally 
how  he/she  feels  the  course  is  going. 

3.  Explain  that  the  assessment  is  part  of  an  important  division  study 
of  the  effect  of  student  ratings  of  instruction  on  teaching,  and  how 
much  his/her  participation  in  the  study  is  appreciated. 

4.  Give  the  instructor  a  copy  of  the  computer  summary  results  and  the 
comparative  feedback  data.   Allow  the  instructor  5-10  minutes  to 
look  over  the  information. 

5.  Tell  the  instructor  that  in  order  to  help  him/her  to  understand  the 
results,  you  will  be  happy  to  review  the  report  with  him/her. 

6.  Proceed  to  summarize  and  interpret  the  results.   Start  with  the 
overall  results,  then  proceed  to  interpret  parts  I,  II,  III,  IV,  and 
V  of  the  report.   Stress  the  positive  item  results  as  well  as  those 
needing  improvement . 

7.  Proceed  to  interpret  the  comparative  data  results  by  commparing  in- 
structor self-evaluation  to  the  overall  student  ratings  and  in- 
dividual items  with  a  focus  on  the  discrepancy  items. 

8.  Interpret  Part  VI,  Diagnostic  Summary  and  Suggestions  for  Instruc- 
tional Improvement. 
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9.  Thank  the  instructor  for  his/her  time.   Remind  him/her  that  there 
will  be  one  more  student  assessment  at  the  end  of  the  term. 
Emphasize  that  the  Division  of  Lifelong  Learning  is  willing  to  help 
the  instructor  with  any  aspect  of  his/her  course. 


APPENDIX  G 
STUDY  COVER  LETTERS 


Dear  Lifelong  Learning  Instructor: 

As  you  are  aware,  the  School  of  Extended  Studies  is  involved 
in  a  very  important, university-wide  study  investigating  new 
techniques  for  gathering  student  ratings  of  instruction  data. 
This  study  is  extremely  vital  since  it  will  give  the  University 
and  the  School  of  Extended  Studies  new  insights  into  the  effect  of 
student  ratings  on  teaching  behavior.  Of  particular  interest  in 
this  investigation  is  what  students  are  able  to  evaluate  in  the 
classroom  and  how'useful  this  information  might  be  to  the  instructor. 
Your  full  participation  is  expected  in  this  study  since  its  results 
are  critical  to  the  future  planning  of  the  Division  of  Lifelong 
Learning. 

Incorporated  into  this  study  will  be  the  "piloting"  of  a  new 
student  evaluation  form  which  will  be  administered  in  each  class 
by  the  course  coordinator.  Most  classes  will  have  this  form 
administered  twice  during  this  semester;  once  at  mid-term  and  once 
at  the  end  of  the  term  and  some  only  at  the  end  of  the  term, 
depending  upon  which  group  you  are  assigned.  In  any  case,  you  have 
my  complete  assurance  that  the  strictest  confidence  will  be  keot, 
and  that  only  you  and  the  course  coordinator  will  see  your  individual 
student  ratings  report. 

On  behalf  of  the  Division  of  Lifelong  Learning  let  me  thank  you 
for  your  participation  in  this  valuable  research  study. 

Sincerely, 


Lee  Leavengood,  Director 
Division  of  Lifelong  Learning 
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Saiat  Leo  College 


Educational  Services 
(904)  538-8236 


Post  Office  Box  2248 
Saint  Leo,  Flonda  33574-2248 


TO: 
FROM: 

RE: 
DATE: 


Educational  Services  Faculty 


Dav 
Ed 


vid  J.  Koval,  Dean  y\(\/^. 
ucational  Services  ^  rjK. 

Consent  to  participate  in 

research  study 

PLEASE  READ  IMMEDIATELY 


January  23,-  1986 


i^t/^ 


Dr   Bernard  S.  Parker,  Vice  President  for  Academic  Affairs,  and 
I  have 'given  Cheryl  Burbano  permission  to  solicit  your  participation 
in  a  graduate  research  study.   Mrs,  Burbano  is  a  Saint  Leo  College 
alumna,  works  at  the  University  of  South  Florida  in  Continuing 
Education,  and  is  doing  her  dissertation  research. 

The  research  study  is  about  student  evaluations  of  instructors. 
Since  we  require  student  evaluation  of  instructors  we  wish  to 
participate  in  this  study  with  the  hope  of  learning  a  more  effective 
way  to  provide  you  with  feedback  for  your  personal  professional 
development . 

The  study  will  be  for  this  current  term  only.   Should  you 
decline  to  participate,  please  call  or  write  us  immediately. 
Otherwise,  we  will  proceed  to  assign  you  to  a  group  and  begin  the 
study. 

Please  be  assured  you  will  receive  feedback  data  from  this 
study.   Likewise,  be  assured  your  participation  is  voluntary. 
All  results  will  be  treated  confidentially. 


APPENDIX  H 

EVALUATION  ADMINSTRATION  INSTRUCTIONS 


RFAD  TO  STUDENTS 

The  School  of  Extended  Studies  (Weekend  College  of  Saint  Leo)  is 
participating  in  an  important  research  study  that  will  last  this  terra 
only.   Involved  in  this  study  is  the  use  of  a  new  student  evaluation 
form  called  IDEA  developed  by  Kansas  State  University.   The  results  of 
this  study  will  be  important  for  future  planning  for  the  Division  of 
Lifelong  Learning  (Weekend  College).   Therefore,  your  input  about  this 
class  is  both  important  and  appreciated. 

Since  this  form  was  designed  to  be  useful  in  many  different  kinds 
of  classes,  not  every  question  applies  equally  to  every  kind  of  class. 
Please  don't  worry  about  this,  since  the  computer  analysis  of  your 
answers  weighs  the  importance  of  each  question  depending  upon  the  kind 
of  course.   Answering  each  question  accurately  and  honestly  will  provide 
the  most  helpful  information. 

Because  of  the  importance  of  this  study,  please  turn  the  card  over 
on  the  back  and  write  the  last  four  digits  of  your  Social  Security  num- 
ber.  If  you  do  not  have  a  Social  Security  number,  write  the  last  four 
letters  of  your  last  name.   This  will  help  the  researcher  code  your  card 
into  the  computer,  while  keeping  your  answers  anonymous.   The  instructor 
will  only  see  the  total  scores  for  each  item  and  not  any  individual 
cards . 

Please  be  sure  to  darken  your  responses  completely  in  the  bubble. 

Any  questions?   Please  begin. 
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APPENDIX  I 
PART-TIME  FACULTY  SELF-RATING  INSTRUCTION  SHEET 


INSTRUCTOR: 

PLEASE  ANSWER  EACH  ITEM  OF  THE  MID -TERM  EVALUATION  AS  YOU 
FEEL  YOUR  STUDENTS  WILL  RATE  YOU  AND  THEMSELVES. 
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IDEA  Survey  ^onr  —  Student  Reactions   U)   Instruction  and  Courses. 
Your  thoughtful   ans-ers   to  these  questions  will   provide  heloful    infomatlon   to  your  ms  tr.:  tar . 


•Describe   the  frequency  of  your   instructor's    teaching  procedures,   using   the 
fol 'owing  code: 
l-HARDLY  EVER  •2-OCCASSIONAJ.LY     3-SOMETIHES     4-FREQUENTLY     5-ALK)ST  AL'«AYS 


The  Instructor: 

1  Proflctes   teacher-student  discussion  (as  opposed  to  mere  responses    to  Questions ). 

2  Finds  -ays  to  help  students  answer   their  own  questions.      ■ 
3'  Encourages  students  to  express  themselves  freely  and  openly. 
4!  Seem  enthusiastic  about  the  subject  matter. 

5  Changes  approaches   to  neet  new  situations. 

6  Gives  exanlnatlons  which  stress  unnecessary  neronzatlon. 

7  Speaks  with  expr«si»en«ss  and  variety  In  tone  of  voice. 

6.  r«iaonstrates   the   laportance  and  significance  of  the  subject  wtter. 

9  Hjnes  presentations  which  are  dry  and  dull. 

IC  Hakes   It  clear  how  each  topic  fUs   Into  the  course. 

li!  Explains  the  reasons  for  criticisms  of  students'   academic  performance. 

12  GIvM  exMlrutlon  questions  which  are  unclear.  .         ,        1.  .., 

13  EfKourages  stu<JMt  co—nts  even  when  th«y  turn  out  to  be  Incorrect  or   Irrelevant 

14  SuMurlzes  aaurlal   In  a  oinner  which  aides  retention.  ,     _  ^  ,     ^     ... 

15  Stimulates  students  to  InUllectual   effort  beyond  that  required  by  -ost  courses. 
16.  Clearly  suted  th«  objectives  of  the  course. 

17  Explains  course  material   clearly,  and  explanations  are  to  the  point. 

18.  Relates  course  material   to  real    life  situations.  /„,,l„, 

19.  Gives  examination  questions  which  are  unreasonably  detailed  (picky). 

20.  Introduces  stlmilatln^  Ideas  about  the  subject. 


On  each  of  the  objectives  listed  belo-.  rate  the  progress  you  art  making  In  t,.1j 
cwSa^d  with  thjl  nade  In  other  courses  you  have  Uken  at  this  university  or  a-^ 
university.       In  this  course  my  progress  so  far  is: 

1-LOW  (lowest  10  percent  of  courses   I  have  taken  here) 

2-LOW  AVERAGE  (next  20  percent  of  courses) 

3-AVEHAGE  (middle  40  percent  of  courses) 

4-HlGH  AVERAGE  (next  20  percent  Of  courses) 

5-HlGH  (highest  10  perc'-nt  of  courses) 


co.-rss 
the- 


>rogress  on: 

21       Gaining  factual   knowledge  (tennlnology.  classifications,  methods,   trends). 
27      Leamlna  fundamenUl   principles,  generalizations,  or  theories. 

23.  L^l^lnj  t^awly  course  material    to  Improve  rational    thinking,   problem-solving  and 

24.  £l!ij?nr!Si'1c  Skins,  coapetencles  and  points  of  view  needed  by  professionals   In 
the  field  most  closely  relattJ  to  this  course.  ,       ,   ,  1.       ,^.4„. 

25.  LMrnlnghw  professionals  in  this   field  go  about  the  process  of  gaining  ne*  knowledge. 

26.  Developing  creative  capacities.  ,.w  H(cr«nMr.l 
27      Developing  a  sense  of  personal   responsibility  (self-reliance,  self  discipline  . 
zi:     Gaining  a  broader  undersunding  and  appreciation  of  Intel  lectual -cul  tural   activity 

(i»js1c.  science,  literature,  etc.). 

29.  Developing  skill   In  expressing  myself  orally  or  In  writing. 

30.  Discovering  the  1«^>11cat1ons  of  the  course  material   for  undersUndfng  iaxifiJ  ^ 
(interests,  talents,  values,  etc.). 


•  On  the  next  four  questions,  compare   this  course  with  others  you  have   t^ken  at  this 
institution,  using  the  foUowIng  code: 

1-MUCH  LESS  THAN  MOST  COURSES     2-LESS  THAN  MOST     3-AflOUT  AVERAGE 
4-WRE  THAN  «ST     5-MUCH  «RE  THAN  «ST 


The  course: 


31:     Amount  of  reading. 

32      A«nunt  of  work  In  other  (non-reading)  asslgwwnts. 

J3:     SIJIl^li^wMcrtircorrsirangs  together  (various  topics  and  class  activities   ar 
related  to  each  other). 
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Describe  your  attitude  and  behavior  In  this  course,  using  the  followlnq  code: 
1-DEFINITELY    2-MORE  FALSE  THfiH  TRUE   3-IN  BETV/EEN    4-MORE  TRUE  TMAN  FALSE 
5-DEFIHITELY  TRUE 


Self-rating  so  far: 


35.  I  have  worked  harder  In  this  course  than  on  most  courses  I  have  taken. 

36.  I  had  a  strong  desire  to  take  this  course. 

37.  I  would  like  to  take  another  course  from  this  Instructor. 

38.  As  a  resuU  of  taking  this  course,  I  haw  nore  positive  feelings  toward  this  field 
of  study. 

39.  Leave  this  space  blank. 

A.  Blacken  space  nunber  1  on  the  Response  Card. 


•For  the  following  six  (questions,  8-6,  describe  the  frequency  of  your  instructor's 
teaching  procedures,  using  the  following  code: 

1-HARDLY  EVER    2-OCCASIONALLY    3-SOMETIMES    4-FREOUENTLY    5-ALMOST  ALWAYS 


The  Instructor: 

8.  Relates  various  topics  and  class  activities  to  each  other. 

C.  Discusses  the  cultural  and  ethetic  values  of  the  subject. 

D.  Relates  the  course  materials  to  other  courses  and  academic  fields. 

E.  Is  willing  to  spend  extra  tine  with  students. 

F.  Cares  about  students  as  people. 

G.  Shows  Interest  In  student  Ideas. 


•Please  answer  these  additional  questions  using  the  following  responses: 
1-STRONGLY  DISAGREE    2-DISAGREE    3-NEITHER  AGREE  OR  DISAGREE    4-AfiREE 
5-STRONGLY  AGREE         ___^ 


40.  Compared  to  others  I  have  had,  this  Instructor  Is  one  of  the  most  effective. 

41.  Overall,  this  course  Is  of  considerable  value  to  me. 
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INSTRUCTOR: 


PLEASE  ANSWER  EACH  ITEM  OF  THE  END-OF-TERM  EVALUATION  AS 
YOU  FEEL  YOUR  STUDENTS  WILL  RATE  YOU  AND  THEMSELVES. 
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SURVEY  FOfW  -  SrUDB¥  R&OKDNS  TO  INSTRUCTION  AND  COURSES 


Your  ihouqhilul  antiran  lo  ItwM  quMllant  »lll  proirtd*  IwtptuI 
Information  (o  your  instructor. 


•  0««erib«  lh«  fr»qu«ncy  ol  your  Intlnjctor'i  tMCMng  pfOC»- 
duroi.  using  tt)«  following  cod«: 

1— HirdlyEvtr  3— SonMIImM 

2— Occ«sion«ily       4— Fr»qu«ntlY        5— Almosl  Alwayt 


oopos«d    to   mere 


The  Instructor 
1   Oromoteo    ^eacrier-student    discussion 
responses  to  questions). 

2.  round  *avs  to  leio  students  answer  ttieir  own  questions. 

3.  Encouragea  students  to  exoress  themselves  Ireely  and  openly. 

4  Seemed  enthusiastic  aoout  the  suOiect  matter. 

5  Changed  aoproacnes  to  meet  new  situations. 

.«  Gave  examinations  wnich  stressed  unnecessary  memorization. 

7  Sooke  with  expressiveness  ar>0  vartery  in  tone  of  voice. 

a  Oemonstraiea  tne  imponance  and  significance  of  the  subject 
matter 

9.  Made  presentations  which  were  dry  and  dull, 
to  Made  It  Clear  now  each  topic  tit  into  the  course. 
II   Explained  tne  reasons  for  criticisms  of  students'  academic  per- 


l2  \3ave  examination  questions  which  were  unclear. 
13.  Encouraged  student  comments  even  when  they  turned  out  to  Oe 
incorrect  or  irrelevant. 

1 4  Summarized  matenal  m  a  manner  which  kided  retention. 

15  Stimulated  students  to  mtellecluai  effort  Beyond  that  required  6y 
most  courses. 

'6.  Clearly  stated  the  oDiectives  of  the  course. 

17  Explained  course  material  clearly,  and  explanations  were  lo  the 

18  ^'eiated  course  material  to  real  life  situations. 

19  Gave  examination  questions  which  were  unreasonaDly  detailed 
ipicxyl. 

20  Introduced  stimulating  ideas  aOout  thesuoiect. 


•  On  each  of  the  ab|ecil«e*  Haled  betow,  rate  the  prograea  you 
have  mede  in  this  courM  compared  with  tlui  made  In  other 
courses  you  have  taken  at  thie  college  or  untvorslty.  In  tht* 
course  my  progress  wee: 

1  —  Low  lloweet  1 0  percent  of  coureee  I  he>e  taken  here) 
2— LOW  Average  (neit  20  percent  ol  eouiwe) 
3— Average  (middle  40  percent  ol  ceureee) 
4— High  Average  Ineit  20  percent  ol  eeurseo) 
5— Hign  (higheet  10  percent  ol  courtea) 


Progress  on: 

21  Gaming  factual  knowledge  (terminology,  cliaailicatlone.  meth- 
ods, trendsi. 

22.  Learning  fundamental  principles,  generalizations,  ortheoriee. 

23.  Learning  to  apply  course  matenal  to  improve  rational  thinking, 
srooiem-soiving  and  decision-making. 

24  Developing  specific  skills,  competencies  and  points  ol  view 
needed  Dy  protessionals  in  the  field  most  closely  related  to  this 
course. 

25.  Learning  how  professionals  in  this  field  go  about  the  process  ol 
gaming  new  knowledge. 

26.  Developing  creative  capacities. 

27.  Developing  a  sense  of  personal  responsibility  (sell-reliance,  sell- 
discipiinei. 

28.  Gaming  a  broader  understanding  and  appreciation  ol  intellectual- 
cultural  activity  (music,  science,  literature,  etc.). 

29  Developing  skill  in  expiessing  myself  orally  or  in  writing. 
30.  Discovering  the  implications  ol  the  course  meienal  for  under- 
standing myself  (interests,  talents,  values,  etc.). 


•  On  the  next  four  questions,  compare  this  course  with  others 
you  he«e  taken  at  this  insiitullon.  using  the  following  code: 

1 —Much  Lees  than  Most  Coursee 

2— Leee  than  Moet 

3— About  Average 

4— More  then  Most 

5— Much  More  then  Moat 


The  Course: 

31  Amount  of  reading. 

32.  Amount  ol  work  in  other  (non-reading)  assignments. 

33.  Difficulty  of  subiect  matter. 

34.  Degree  to  which  tne  course  hung  together  ivarious  topics  and 
class  activities  were  related  to  each  other). 


•  Oeecrtbe  your  ettltudee  toward  and  behevlor  in  this  course. 

using  the  lollowtng  code: 
1— OellnitelyFelee 

2— Mere  False  than  True      4— MoreTniethan  Falee 
3— In  Beheeen  S— Oellnitely  True 


Sell-reling; 

35.  I  worked  harder  on  this  course  than  on  most  c 
36. 1  had  a  Strang  desire  to  take  this  course. 

37.  I  would  like  to  take  another  course  (romthisinstructor. 

38.  Aa  a  result  of  taking  this  course,  I  have  more  positive  'eeii 
toward  this  field  of  study 

39.  Leave  this  space  oiank.  Continue  with  question  A. 

A.  Blacken  space  number  i  on  the  Response  Card. 


•  For  the  following  six  queallone.  B-Q.  deeeribe  the  Irequency 

ol  your  ineiructor's 

leeching  procedures 

using  the  following 

code: 

t-HerdlyEver 

3— Sometlmee 

2— Occasionally 

4— Frequently 

S— Almoet  Aheeye 

The  Inetiuctor 

B.  Related  venous  topics  and  class  activities  to  eecn  other. 

C.  Discussed  the  cultural  and  eetnetic  values  of  the  subiect. 

0.  Related  the  course  matenais  to  other  courses  and  academic 

fields. 
E.  Was  willing  to  spend  extra  time  with  students. 
F  Cared  about  students  as  people. 
G.  Showed  interest  in  student  ideas. 


please  answer  triese  addUl 
following   responses: 


questions  using  the 


1  -  Strongly  Disagree 

2  -  Disagree 

3  -  Neither  Agree  or  Disagree 

4  .    Agree 

5  -   Strongly  Agree 


Overall,  this  course  was  of  considerable  value  to 


APPENDIX  J 
[PEA  FACULTY  INFORMATION  CARDS  INSTRUCTION  SHEET 


DIRECTIONS  FOR  COMPLETING  INFORMATION  CARD  II 

F;^rii1t.v  Information  Card  II  (purple):  Provides  space  for  information  about 
the  course,  ratings  of  course  objectives,  and  the  ootional  response  section. 
USE  42   PENCIL  ONLY. 

1   Department  Code,  Course  number,  hours,  days  and  number  enrolled  will  be 
filled  in  for  you  PLEASE  DO  NOT  FILL  IN. 

2.  Print  your  name  on  the  back  of  the  card. 

3.  Objectives:  Ten  objectives  are  listed  below.  Rate  each  of  them  as: 
"Of  no  more  than  Minor  Importance'  (M),  "Imoortant"  (I),  or  "Essential" 
(E)  by  blackening  the  appropriate  letter  on  Card  II.  We  strongly 
recommend  that  you  DO  NOT  select  more  than  a  total  of  3-5  objectives 
as  "Essential"  or  "Important."  If  you  have  questions,  contact  your 
Lifelong  Learning  coordinator. 

1)  Gaining  factual  knowledge  (terminology,  classifications,  methods, 

trends)  ,  .  ^.        ■ 

2)  Learning  fundamental   principles,  generalizations,  or  theories 

3)  Learning  to  apply  course  material    to  improve  rational    thinking, 
problem-solving  and  decision-making 

4)  Developing  specific  skills,  competencies  and  points  of  view 
needed  by  professionals  in  the  field  most  closely  releated  to 
this  course  ^         ^,  ^ 

5)  Learning  how  professionals  in  this  field  go  about  the  process  ot 
of  gaining  new  knowledge 

6)  Developing  creative  capacities  ,     ,,      ,. 

7)  Developing  a  sense  of  personal   responsibility  (self-reliance, 

self-discipline)  .      .         .  ■   ^  ■,-,     ^.     t 

8)  Gaining  a  broader  understanding  and  aporeciation  of  mtel lectua i - 
cultural   activity  (music,  science,   literature,  etc.)         ^ 

9)  Developing  skill   in  expressing  oneself  orally  or  in  writing 

10)     Discovering  the  implications  of  the  course  material    for  understanding 
oneself  (interests,  talents,  values,  etc.) 

4       Research  Questions:     Use  response  columns  A-H  for  your  responses  to  re- 
search questions.     These  questions  are  for  research  purposes  only  and 
allow  us  to  study  their  relationship  to  student  ratings. If  a  question 
does  not  seem  to  apply  or  you  do  not  wish  to  answer,  simply  leave  it 
blank.     However,  be  certain  to  darken  completely  the  numerical   space 
below  the  alphabetic  item  for  those  answered. 

A.     Which  of  the  following  circumstance  apply  to  the  decision  to  have 
this  specific  course  rated? 
1)     Entirely  your  (the  instructor's)  decision 
2       Informal   decision  of  two  or  more  instructors,  e.g.,  team 

taught  course  or  arouo  of  instructors  teaching  survey  course. 
3)     You  were  required"  to  have  one  (or  more)   course(s)   rated,  but 
you  (the  instructor)   chose  the  course 
You  were  required  to  have  this  specific  course  rated 
You  were  required  to  have  all   courses  rated 
Other 


4) 
5) 
6) 

What  is  the  primary  purpose  for  obtaining  these  ratings? 
1)  Will  be  used  for  administrative  purposes  (promotion,  salary 
increase,  etc.)  at  option  of  the  administration 
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2)  May  be  used  for  administrative  purposes,  at  option  of 
the  instructor 

3)  For  use  of  students  in  selecting  courses  and/or 
instructors 

4)  As  data  for  a  research  project 

5)  Solely  for  the  use  of  the  instructor  as  information 
about  instruction 

6)  Other 

C.  What  is  the  secondary  purpose  for  obtaining  the  ratings? 
(Use  the  same  options  as  question  B.  If  there  is  no 
secondary  purpose,  leave  blank)  

D.  How  many  times  have  you  had  this  specific  course  rated  using 
IDEA?  . 

E.  How  many  times  have  you  had  any  course  rated  using  IDEA?  

F.  For  how  many  academic  years  have  you  used  IDEA?  (E.g.,  if  this 
is  you  first  year  mark  "1 ") 

G.  What  importance  do  you  give  to  teaching  compared  to  research? 

1)  Teaching  definitely  more  important 

2)  Teaching  somewhat  more  important 

3)  Both  equally  important 

4)  Research  somewhat  more  important 

5)  Research  definitely  more  important 

H   What  importance  is  given  to  teaching  compared  to  research  by  those 
who  have  the  most  weight  in  making  the  personnel  decisions  which 
affect  you?  (Use  the  same  options  as  for  question  G) 

5   Academic  Rank:  In  the  space  below  the  ratings  of  the  objectives,  please 
indicate  your  academic  rank  by  marking  the  appropriate  response. 


APPENDIX  K 
FINAL  COURSE  EVALUATION  FOLLOW-UP  LETTER 
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UNIVERSITY    OF    SOUTH     FLORIDA 

TAMPA    •    ST.  PETERSBURG     •    FORT  MYERS    •    SARASOTA 


SCHOOL  OF   EXTENDED  STUDIES 
DIVISION   OF   LIFELONG   LEARNING 
TAMPA,    FLORIDA    33620 

813      974  2403 


MEMORANDUM 

Date:         April ,  1986 

To: 

From:   Cheryl  M.  Burbano,  Lifelong  Learning  Coordinator 

Subject:  Final  Course  Evaluation 


As  you  are  aware,  the  School  of  Extended  Studies  at  the  University  of 
South  Florida  is  involved  in  an  important  research  project  this  term. 
This  study  utilizes  a  new  kind  of  student  evaluation  form  called  IDEA 
developed  by  Kansas  State  University.  The  results  of  this  study  will 
be  important  for  future  planning  for  the  Division  of  Lifelong  Learning. 
Therefore,  your  input  about  this  class  is  both  important  and  appreciated. 

Since  this  form  was  designed  to  be  useful  in  many  different  kinds  of 
classes,  not  every  question  applies  equally  to  every  kind  of  class.  Please 
do  not  worry  about  this,  since  the  computer  analysis  of  your  answers 
weighs  the  importance  of  each  question  depending  upon  the  kind  of  course. 
Answering  each  question  accurately  and  honestly  will  provide  the  most 
helpful  information. 

Please  complete  the  enclosed  form  by  placing  your  responses  on  the  Student 
Response  Card  (blue  card).  Please  use  pencil  only.  Return  your  completed 
blue  card  in  the  enclosed  stamped  envelope  as  soon  as  possible. 

Thank  you  for  your  help  and  participation  in  evaluating  your  class  and  in 
this  research  project. 


APPENDIX  L 

COMPUTER  PRINTED  IDEA  SUMMARY  REPORT,  INTERPRETATION  GUIDELINES, 
AND  INSTRUCTOR  SELF-RATINGS  TO  STUDENT  RATINGS  COMPARISON 
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UNIVERSITY  OF  SOUTH  FLORIDA 
Division  of  Lifelong  Learning 


Instr : 


Evaluation  Date: 
IDEA  Mid-Term  Evaluation  Report  by  Item 


Student  Eval . 

Instructor 

Item  No. 

Mean 

Self- Evaluation 

21. 

22. 

PART  I   24. 

PRO-     25. 

GRESS 

23. 

RATINGS 

26. 

29. 

27. 

28. 

30. 

PART  II  31. 

COURSE   32. 

DESCRIP-33. 

TION 

34. 

35. 

PART  III 

36. 

STUDENTS' 

37. 

SELF- 

RATING   38. 

Difference 
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Student  Eval . 

Instructor 

Item 

No. 

Mean 

Self -Evaluation 

1. 
2. 

3. 
5. 

11. 

PART  IV 

13. 

TEACHING 

8. 

METHODS 

10. 

14. 

16. 

17. 

18. 

4. 

7. 

(9). 

15. 

20. 

(6) 
(12) 

(19) 

PART  V 

40. 

EXTRA 

QUESTIONS  41. 

Difference 


*  (  )  Refers  for  items  where  low  scores  are  desirable 


APPENDIX  M 
LEARNING  CHARACTERISTICS  OF  ADULT  STUDENTS 


LEARNING  CHARACTERISTICS  OF  ADULT  STUDENTS 


The  following  is  a  list  of  mature  student  learning  characteristics: 


1.  Adult  students  are  a  self -selected  group  and  are  in  your  course 
because  they  want  to  be  there. 

2.  Adults  prefer  instructor-centered  instructions  in  which  the 
instructor  controls  the  organization  and  content  of  each  class 
meeting.   This  preference  is  usually  due  to  conditioning,  and 
results  in  a  feeling  that  informal  teacher-student  sessions  are 
usually  a  waste  of  time. 

3.  Adult  students  prefer  a  controlled  reciprocal  relationship  with 
an  instructor,  where  they  can  share  pertinent  knowledge  and  ex- 
perience that  they  as  students  bring  to  a  course.   It  is  important 
that  adult  students  who  possess  accumulated  experience  be  recog- 
nized. 

4.  Adults  are  interested  in  task- centered  learning;  thus  the  relevance 
of  the  course  material  should  be  made  very  clear  to  them.   They  want 
to  be  able  to  see  how  they  will  use  the  information  they  are  learn- 
ing. 

5.  Despite  a  preference  for  instructor-centered  instruction,  adult 
students  prefer  an  informal  learning  atmosphere  where  they  can  feel 
comfortable  asking  questions  and  making  comments. 
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APPENDIX  N 


COVER  MEMO  FOR  MID -TERM  EVALUATION  SUMMARY  EXPLANATION 
(PRINTED  FEEDBACK  ONLY  GROUP) 
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UNIVERSITY    OF    SOUTH     FLORIDA 

TAMPA    •    ST.  PETERSBURG     •     FORT  MYERS    •    SARASOTA 


SCHOOL  Of  EXTENDED  STUDIES 
DIVISION  OF  CONFERENCES  It  INSTITUTES 
TAMPA,   FLORIDA   336J0 

813:    97«'2403 


March/April,    1986 


MEMORANDUM 


TO: 

FROM:      Cheryl  M.  Burbano,  Lifelong  Learning  Coordinator 

SUBJECT:    Mid-Term  Evaluation  Summary 


Please  find  enclosed  your  copy  of  the  IDEA  Report  of  your  mid-term  student 
evaluation  of  instruction.  All  interpretative  data  is  found  on  the 
backside.  Also  enclosed  Is  an  additional  report  which  compares  the  mean 
student  rating  (average)  you  received  on  each  individual  item  to  your  own 
self-evaluation.  Included  in  this  report  is  a  difference  score  for 
each  item  showing  the  difference  of  your  average  student  rating  minus  your 
self-rating.  Therefore,  a  negative  difference  score  indicates  a  discrepancy 
between  your  perceived  teaching  behavior  and  how  the  students  actually 
rated  your  behavior. 

The  final  student  evaluation  for  your  course  is  scheduled  for  

Please  contact  me  at  974-2403  if  this  needs  to  be  re-scheduled. 


APPENDIX  0 
ANOVA  TABULAR  SUMMARY  OF  RESULTS  OF  HYPOTHESIS  1 
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APPENDIX  P 
ANOVA  TABULAR  SUMMARY  OF  RESULTS  OF  HYPOTHESIS  5 
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BIOGRAPHICAL  SKETCH 
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